[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:22:25 dist-test-slave-5k54 krb5kdc[23852](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:22:25 dist-test-slave-5k54 krb5kdc[23852](info): set up 2 sockets
Aug 08 18:22:25 dist-test-slave-5k54 krb5kdc[23852](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260808 18:22:26.541980 23842 mini_kdc.cc:121] Time spent starting KDC: real 1.072s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Aug 08 18:22:26 dist-test-slave-5k54 krb5kdc[23852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:22:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:22:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.72.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
I20260808 18:22:26.671021 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:42745
--webserver_interface=127.23.72.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:41521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:42745
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:26.795454 23865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:26.795809 23865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:26.795898 23865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:26.799929 23865 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:22:26.800030 23865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:26.800063 23865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:26.800105 23865 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:22:26.800141 23865 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:22:26.805483 23865 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:41521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:42745
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:42745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.72.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23865
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:26.806660 23865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:26.807753 23865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:26.816629 23872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:26.816636 23874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:26.816669 23871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:26.817034 23865 server_base.cc:1034] running on GCE node
I20260808 18:22:26.817744 23865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:26.818958 23865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:26.820139 23865 hybrid_clock.cc:648] HybridClock initialized: now 1786213346820111 us; error 40 us; skew 500 ppm
Aug 08 18:22:26 dist-test-slave-5k54 krb5kdc[23852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213346, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:26.823014 23865 init.cc:377] Logged in from keytab as kudu/127.23.72.190@KRBTEST.COM (short username kudu)
I20260808 18:22:26.824077 23865 webserver.cc:460] Webserver started at http://127.23.72.190:46081/ using document root <none> and password file <none>
I20260808 18:22:26.824662 23865 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:26.824728 23865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:26.824999 23865 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:26.827150 23865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "d8ef83464b1a46d2b2f17ab14dfe669a"
format_stamp: "Formatted at 2026-08-08 18:22:26 on dist-test-slave-5k54"
server_key: "7484d4704b970de65f1cbc506d68cbb2"
server_key_iv: "8c7c9ada3a57327a42c296ed6ceb8965"
server_key_version: "encryptionkey@0"
I20260808 18:22:26.827895 23865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "d8ef83464b1a46d2b2f17ab14dfe669a"
format_stamp: "Formatted at 2026-08-08 18:22:26 on dist-test-slave-5k54"
server_key: "7484d4704b970de65f1cbc506d68cbb2"
server_key_iv: "8c7c9ada3a57327a42c296ed6ceb8965"
server_key_version: "encryptionkey@0"
I20260808 18:22:26.831894 23865 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 18:22:26.834504 23883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:26.835914 23865 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:22:26.836066 23865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "d8ef83464b1a46d2b2f17ab14dfe669a"
format_stamp: "Formatted at 2026-08-08 18:22:26 on dist-test-slave-5k54"
server_key: "7484d4704b970de65f1cbc506d68cbb2"
server_key_iv: "8c7c9ada3a57327a42c296ed6ceb8965"
server_key_version: "encryptionkey@0"
I20260808 18:22:26.836192 23865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:26.849762 23865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:26.852806 23865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:26.853034 23865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:26.861168 23865 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.190:42745
I20260808 18:22:26.861193 23940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.190:42745 every 8 connection(s)
I20260808 18:22:26.862213 23865 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:22:26.865262 23941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:26.868470 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 23865
I20260808 18:22:26.868664 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:22:26.870462 23842 external_mini_cluster.cc:1428] Setting key 5eaefe5a61bd27cc7536967a4742e198
I20260808 18:22:26.872720 23941 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8ef83464b1a46d2b2f17ab14dfe669a: Bootstrap starting.
I20260808 18:22:26.875797 23941 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8ef83464b1a46d2b2f17ab14dfe669a: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:26.876703 23941 log.cc:824] T 00000000000000000000000000000000 P d8ef83464b1a46d2b2f17ab14dfe669a: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:26.878757 23941 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8ef83464b1a46d2b2f17ab14dfe669a: No bootstrap required, opened a new log
I20260808 18:22:26.883332 23941 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8ef83464b1a46d2b2f17ab14dfe669a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8ef83464b1a46d2b2f17ab14dfe669a" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 42745 } }
I20260808 18:22:26.883632 23941 raft_consensus.cc:374] T 00000000000000000000000000000000 P d8ef83464b1a46d2b2f17ab14dfe669a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:26.883709 23941 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8ef83464b1a46d2b2f17ab14dfe669a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8ef83464b1a46d2b2f17ab14dfe669a, State: Initialized, Role: FOLLOWER
I20260808 18:22:26.884130 23941 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8ef83464b1a46d2b2f17ab14dfe669a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8ef83464b1a46d2b2f17ab14dfe669a" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 42745 } }
I20260808 18:22:26.884248 23941 raft_consensus.cc:388] T 00000000000000000000000000000000 P d8ef83464b1a46d2b2f17ab14dfe669a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:22:26.884341 23941 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8ef83464b1a46d2b2f17ab14dfe669a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:22:26.884451 23941 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8ef83464b1a46d2b2f17ab14dfe669a [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:26.885381 23941 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8ef83464b1a46d2b2f17ab14dfe669a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8ef83464b1a46d2b2f17ab14dfe669a" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 42745 } }
I20260808 18:22:26.885771 23941 leader_election.cc:302] T 00000000000000000000000000000000 P d8ef83464b1a46d2b2f17ab14dfe669a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d8ef83464b1a46d2b2f17ab14dfe669a; no voters:
I20260808 18:22:26.886188 23941 leader_election.cc:288] T 00000000000000000000000000000000 P d8ef83464b1a46d2b2f17ab14dfe669a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:22:26.886332 23946 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8ef83464b1a46d2b2f17ab14dfe669a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:22:26.886596 23946 raft_consensus.cc:686] T 00000000000000000000000000000000 P d8ef83464b1a46d2b2f17ab14dfe669a [term 1 LEADER]: Becoming Leader. State: Replica: d8ef83464b1a46d2b2f17ab14dfe669a, State: Running, Role: LEADER
I20260808 18:22:26.886902 23946 consensus_queue.cc:237] T 00000000000000000000000000000000 P d8ef83464b1a46d2b2f17ab14dfe669a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8ef83464b1a46d2b2f17ab14dfe669a" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 42745 } }
I20260808 18:22:26.887557 23941 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8ef83464b1a46d2b2f17ab14dfe669a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:22:26.888216 23947 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8ef83464b1a46d2b2f17ab14dfe669a [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8ef83464b1a46d2b2f17ab14dfe669a. Latest consensus state: current_term: 1 leader_uuid: "d8ef83464b1a46d2b2f17ab14dfe669a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8ef83464b1a46d2b2f17ab14dfe669a" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 42745 } } }
I20260808 18:22:26.888351 23947 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8ef83464b1a46d2b2f17ab14dfe669a [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:26.889091 23949 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8ef83464b1a46d2b2f17ab14dfe669a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d8ef83464b1a46d2b2f17ab14dfe669a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8ef83464b1a46d2b2f17ab14dfe669a" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 42745 } } }
I20260808 18:22:26.889261 23949 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8ef83464b1a46d2b2f17ab14dfe669a [sys.catalog]: This master's current role is: LEADER
Aug 08 18:22:26 dist-test-slave-5k54 krb5kdc[23852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
W20260808 18:22:26.892952 23967 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d8ef83464b1a46d2b2f17ab14dfe669a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:22:26.893030 23967 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:22:26.893090 23957 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:22:26.893919 23957 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:22:26.900190 23957 catalog_manager.cc:1347] Generated new cluster ID: 8da8a8edb438429f937fa3cebbda356b
I20260808 18:22:26.900216 23944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:26.875180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42062 (local address 127.23.72.190:42745)
0808 18:22:26.876490 (+ 1310us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:26.876511 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:26.877572 (+ 1061us) server_negotiation.cc:407] Connection header received
0808 18:22:26.881189 (+ 3617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:26.881211 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:26.881602 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:26.881957 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:26.884378 (+ 2421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:26.885450 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:26.887288 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:26.888058 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:26.888229 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:26.893211 (+ 4982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:26.893216 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:26.893228 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:26.893255 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:26.895040 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:26.895636 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:26.895641 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:26.895644 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:26.895751 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:26.897639 (+ 1888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:26.897642 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:26.897644 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:26.898188 (+ 544us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:26.898394 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:26.898974 (+ 580us) server_negotiation.cc:299] Negotiation successful
0808 18:22:26.899244 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1155,"thread_start_us":249,"threads_started":1}
I20260808 18:22:26.900274 23957 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:22:26.914623 23957 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:22:26.915627 23957 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:22:26.924031 23957 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d8ef83464b1a46d2b2f17ab14dfe669a: Generated new TSK 0
I20260808 18:22:26.924783 23957 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.72.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab.
Entry for principal kudu/127.23.72.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab.
I20260808 18:22:26.962651 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.129:0
--local_ip_for_outbound_sockets=127.23.72.129
--webserver_interface=127.23.72.129
--webserver_port=0
--tserver_master_addrs=127.23.72.190:42745
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab
--principal=kudu/127.23.72.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:41521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:27.076815 23972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:27.077072 23972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:27.077148 23972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:27.081296 23972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:27.081360 23972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:27.081445 23972 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.129
I20260808 18:22:27.086725 23972 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:41521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab
--principal=kudu/127.23.72.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.72.129
--webserver_port=0
--tserver_master_addrs=127.23.72.190:42745
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23972
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.129
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:27.088025 23972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:27.089215 23972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:27.098839 23980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:27.098860 23977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:27.098860 23978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:27.099041 23972 server_base.cc:1034] running on GCE node
I20260808 18:22:27.099701 23972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:27.100378 23972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:27.101568 23972 hybrid_clock.cc:648] HybridClock initialized: now 1786213347101542 us; error 52 us; skew 500 ppm
Aug 08 18:22:27 dist-test-slave-5k54 krb5kdc[23852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213347, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:27.104593 23972 init.cc:377] Logged in from keytab as kudu/127.23.72.129@KRBTEST.COM (short username kudu)
I20260808 18:22:27.105592 23972 webserver.cc:460] Webserver started at http://127.23.72.129:44407/ using document root <none> and password file <none>
I20260808 18:22:27.106113 23972 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:27.106173 23972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:27.106338 23972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:27.108471 23972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-0/data/instance:
uuid: "cd70410b81634c54b1241998035effe8"
format_stamp: "Formatted at 2026-08-08 18:22:27 on dist-test-slave-5k54"
server_key: "8ffee28f860ea67c632713ccabbf816f"
server_key_iv: "2ee51f7494aeffe0a6b7fc4d6290783c"
server_key_version: "encryptionkey@0"
I20260808 18:22:27.109103 23972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance:
uuid: "cd70410b81634c54b1241998035effe8"
format_stamp: "Formatted at 2026-08-08 18:22:27 on dist-test-slave-5k54"
server_key: "8ffee28f860ea67c632713ccabbf816f"
server_key_iv: "2ee51f7494aeffe0a6b7fc4d6290783c"
server_key_version: "encryptionkey@0"
I20260808 18:22:27.112998 23972 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 18:22:27.115417 23987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:27.116775 23972 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:22:27.116912 23972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-0/wal
uuid: "cd70410b81634c54b1241998035effe8"
format_stamp: "Formatted at 2026-08-08 18:22:27 on dist-test-slave-5k54"
server_key: "8ffee28f860ea67c632713ccabbf816f"
server_key_iv: "2ee51f7494aeffe0a6b7fc4d6290783c"
server_key_version: "encryptionkey@0"
I20260808 18:22:27.117040 23972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:27.142395 23972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:27.145563 23972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:27.145771 23972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:27.146430 23972 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:22:27.147269 23972 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:22:27.147341 23972 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:27.147445 23972 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:22:27.147500 23972 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:27.156932 23972 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.129:37789
I20260808 18:22:27.156958 24107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.129:37789 every 8 connection(s)
I20260808 18:22:27.157799 23972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
I20260808 18:22:27.159799 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 23972
I20260808 18:22:27.159909 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance
I20260808 18:22:27.160188 23842 external_mini_cluster.cc:1428] Setting key a5d4c8a5ac248c56490d39e68195ab45
Aug 08 18:22:27 dist-test-slave-5k54 krb5kdc[23852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213347, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.72.130@KRBTEST.COM; defaulting to no policy
I20260808 18:22:27.172988 23944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:27.159616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:53553 (local address 127.23.72.190:42745)
0808 18:22:27.159774 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:27.159779 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:27.160669 (+ 890us) server_negotiation.cc:407] Connection header received
0808 18:22:27.162307 (+ 1638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:27.162311 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:27.162363 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:27.162452 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:27.163819 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:27.164594 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:27.166225 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:27.166473 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:27.166585 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:27.169144 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:27.169151 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:27.169156 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:27.169188 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:27.170974 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:27.171553 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:27.171559 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:27.171562 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:27.171661 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:27.172002 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:27.172005 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:27.172006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:27.172187 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:27.172261 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:27.172747 (+ 486us) server_negotiation.cc:299] Negotiation successful
0808 18:22:27.172859 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 18:22:27.173835 24110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:27.160026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:42745 (local address 127.23.72.129:53553)
0808 18:22:27.160519 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:27.160570 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:27.162076 (+ 1506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:27.162643 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:27.162650 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:27.163072 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:27.163660 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:27.163677 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:27.164732 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:27.164738 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:27.166058 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:27.166068 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:27.166626 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:27.166630 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:27.166817 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:27.167473 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:27.167498 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:27.168966 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:27.171103 (+ 2137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:27.171107 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:27.171119 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:27.171405 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:27.171750 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:27.171752 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:27.171753 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:27.171900 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:27.172297 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:27.172302 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:27.172532 (+ 230us) client_negotiation.cc:769] Sending connection context
0808 18:22:27.172681 (+ 149us) client_negotiation.cc:240] Negotiation successful
0808 18:22:27.172959 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":131,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.130@KRBTEST.COM" created.
I20260808 18:22:27.175323 24108 heartbeater.cc:347] Connected to a master server at 127.23.72.190:42745
I20260808 18:22:27.175591 24108 heartbeater.cc:464] Registering TS with master...
I20260808 18:22:27.176129 24108 heartbeater.cc:511] Master 127.23.72.190:42745 requested a full tablet report, sending...
I20260808 18:22:27.177624 23901 ts_manager.cc:194] Registered new tserver with Master: cd70410b81634c54b1241998035effe8 (127.23.72.129:37789)
I20260808 18:22:27.178787 23901 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.129@KRBTEST.COM'} at 127.23.72.129:53553
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab.
Entry for principal kudu/127.23.72.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab.
I20260808 18:22:27.190891 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.130:0
--local_ip_for_outbound_sockets=127.23.72.130
--webserver_interface=127.23.72.130
--webserver_port=0
--tserver_master_addrs=127.23.72.190:42745
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab
--principal=kudu/127.23.72.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:41521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:27.302632 24113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:27.302873 24113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:27.302917 24113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:27.306954 24113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:27.307008 24113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:27.307097 24113 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.130
I20260808 18:22:27.312361 24113 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:41521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab
--principal=kudu/127.23.72.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.72.130
--webserver_port=0
--tserver_master_addrs=127.23.72.190:42745
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24113
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.130
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:27.313359 24113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:27.314369 24113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:27.323931 24121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:27.323957 24118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:27.323957 24119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:27.324110 24113 server_base.cc:1034] running on GCE node
I20260808 18:22:27.324533 24113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:27.325212 24113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:27.326383 24113 hybrid_clock.cc:648] HybridClock initialized: now 1786213347326349 us; error 45 us; skew 500 ppm
Aug 08 18:22:27 dist-test-slave-5k54 krb5kdc[23852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213347, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:27.329305 24113 init.cc:377] Logged in from keytab as kudu/127.23.72.130@KRBTEST.COM (short username kudu)
I20260808 18:22:27.330178 24113 webserver.cc:460] Webserver started at http://127.23.72.130:34631/ using document root <none> and password file <none>
I20260808 18:22:27.330754 24113 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:27.330853 24113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:27.331084 24113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:27.333133 24113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-1/data/instance:
uuid: "97948bd0d86b4aa69a64e66a40a2030b"
format_stamp: "Formatted at 2026-08-08 18:22:27 on dist-test-slave-5k54"
server_key: "f13049597ffc2ec5e722c7e51ff1173e"
server_key_iv: "b209189a2eb5bd9f5c3a95762e402aab"
server_key_version: "encryptionkey@0"
I20260808 18:22:27.333690 24113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance:
uuid: "97948bd0d86b4aa69a64e66a40a2030b"
format_stamp: "Formatted at 2026-08-08 18:22:27 on dist-test-slave-5k54"
server_key: "f13049597ffc2ec5e722c7e51ff1173e"
server_key_iv: "b209189a2eb5bd9f5c3a95762e402aab"
server_key_version: "encryptionkey@0"
I20260808 18:22:27.337735 24113 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 18:22:27.340358 24130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:27.341599 24113 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:22:27.341728 24113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-1/wal
uuid: "97948bd0d86b4aa69a64e66a40a2030b"
format_stamp: "Formatted at 2026-08-08 18:22:27 on dist-test-slave-5k54"
server_key: "f13049597ffc2ec5e722c7e51ff1173e"
server_key_iv: "b209189a2eb5bd9f5c3a95762e402aab"
server_key_version: "encryptionkey@0"
I20260808 18:22:27.341837 24113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:27.386211 24113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:27.390019 24113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:27.390185 24113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:27.390832 24113 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:22:27.391765 24113 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:22:27.391825 24113 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:27.391898 24113 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:22:27.391938 24113 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:27.402053 24113 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.130:45889
I20260808 18:22:27.402077 24250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.130:45889 every 8 connection(s)
I20260808 18:22:27.403043 24113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
I20260808 18:22:27.407804 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 24113
I20260808 18:22:27.407934 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance
I20260808 18:22:27.408182 23842 external_mini_cluster.cc:1428] Setting key db1a637355d604efcd08edcf35db3d14
Aug 08 18:22:27 dist-test-slave-5k54 krb5kdc[23852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213347, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.130@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:22:27.418139 23944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:27.404870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:60055 (local address 127.23.72.190:42745)
0808 18:22:27.405021 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:27.405025 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:27.405887 (+ 862us) server_negotiation.cc:407] Connection header received
0808 18:22:27.407260 (+ 1373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:27.407264 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:27.407324 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:27.407467 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:27.408936 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:27.409514 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:27.410651 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:27.411087 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:27.411225 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:27.414297 (+ 3072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:27.414300 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:27.414302 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:27.414327 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:27.415911 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:27.416459 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:27.416464 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:27.416466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:27.416562 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:27.416968 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:27.416974 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:27.416976 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:27.417169 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:27.417283 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:27.417818 (+ 535us) server_negotiation.cc:299] Negotiation successful
0808 18:22:27.417956 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
WARNING: no policy specified for kudu/127.23.72.131@KRBTEST.COM; defaulting to no policy
I20260808 18:22:27.419039 24254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:27.405268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:42745 (local address 127.23.72.130:60055)
0808 18:22:27.405730 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:27.405774 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:27.407045 (+ 1271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:27.407659 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:27.407667 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:27.408137 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:27.408773 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:27.408800 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:27.409661 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:27.409664 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:27.410521 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:27.410530 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:27.411261 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:27.411264 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:27.411495 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:27.412208 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:27.412234 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:27.414118 (+ 1884us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:27.416036 (+ 1918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:27.416042 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:27.416058 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:27.416347 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:27.416663 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:27.416667 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:27.416670 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:27.416861 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:27.417302 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:27.417310 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:27.417561 (+ 251us) client_negotiation.cc:769] Sending connection context
0808 18:22:27.417764 (+ 203us) client_negotiation.cc:240] Negotiation successful
0808 18:22:27.418103 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":114,"threads_started":1}
I20260808 18:22:27.420411 24251 heartbeater.cc:347] Connected to a master server at 127.23.72.190:42745
I20260808 18:22:27.420624 24251 heartbeater.cc:464] Registering TS with master...
I20260808 18:22:27.421211 24251 heartbeater.cc:511] Master 127.23.72.190:42745 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.131@KRBTEST.COM" created.
I20260808 18:22:27.422469 23901 ts_manager.cc:194] Registered new tserver with Master: 97948bd0d86b4aa69a64e66a40a2030b (127.23.72.130:45889)
I20260808 18:22:27.423353 23901 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.130@KRBTEST.COM'} at 127.23.72.130:60055
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab.
Entry for principal kudu/127.23.72.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab.
I20260808 18:22:27.439165 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.131:0
--local_ip_for_outbound_sockets=127.23.72.131
--webserver_interface=127.23.72.131
--webserver_port=0
--tserver_master_addrs=127.23.72.190:42745
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab
--principal=kudu/127.23.72.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:41521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:27.551126 24257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:27.551404 24257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:27.551501 24257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:27.555495 24257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:27.555570 24257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:27.555656 24257 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.131
I20260808 18:22:27.560595 24257 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:41521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab
--principal=kudu/127.23.72.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.72.131
--webserver_port=0
--tserver_master_addrs=127.23.72.190:42745
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24257
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.131
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:27.561685 24257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:27.562667 24257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:27.572388 24266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:27.572383 24262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:27.572386 24264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:27.572588 24257 server_base.cc:1034] running on GCE node
I20260808 18:22:27.572986 24257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:27.573578 24257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:27.574738 24257 hybrid_clock.cc:648] HybridClock initialized: now 1786213347574722 us; error 29 us; skew 500 ppm
Aug 08 18:22:27 dist-test-slave-5k54 krb5kdc[23852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213347, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:27.577715 24257 init.cc:377] Logged in from keytab as kudu/127.23.72.131@KRBTEST.COM (short username kudu)
I20260808 18:22:27.578578 24257 webserver.cc:460] Webserver started at http://127.23.72.131:43139/ using document root <none> and password file <none>
I20260808 18:22:27.579119 24257 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:27.579180 24257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:27.579396 24257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:27.581391 24257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-2/data/instance:
uuid: "a78f5aaacd4c45ca819be17b54450c79"
format_stamp: "Formatted at 2026-08-08 18:22:27 on dist-test-slave-5k54"
server_key: "ec54a734916aa6c62b9981f4f7a289d0"
server_key_iv: "c34d7dfe10ad0c0b5803ff22b2ed47da"
server_key_version: "encryptionkey@0"
I20260808 18:22:27.581961 24257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance:
uuid: "a78f5aaacd4c45ca819be17b54450c79"
format_stamp: "Formatted at 2026-08-08 18:22:27 on dist-test-slave-5k54"
server_key: "ec54a734916aa6c62b9981f4f7a289d0"
server_key_iv: "c34d7dfe10ad0c0b5803ff22b2ed47da"
server_key_version: "encryptionkey@0"
I20260808 18:22:27.585695 24257 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260808 18:22:27.588101 24273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:27.589571 24257 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 18:22:27.589673 24257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-2/wal
uuid: "a78f5aaacd4c45ca819be17b54450c79"
format_stamp: "Formatted at 2026-08-08 18:22:27 on dist-test-slave-5k54"
server_key: "ec54a734916aa6c62b9981f4f7a289d0"
server_key_iv: "c34d7dfe10ad0c0b5803ff22b2ed47da"
server_key_version: "encryptionkey@0"
I20260808 18:22:27.589762 24257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:27.609812 24257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:27.613118 24257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:27.613272 24257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:27.613796 24257 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:22:27.614554 24257 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:22:27.614602 24257 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:27.614640 24257 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:22:27.614655 24257 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:27.624670 24257 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.131:37233
I20260808 18:22:27.624696 24397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.131:37233 every 8 connection(s)
I20260808 18:22:27.625593 24257 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
I20260808 18:22:27.625926 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 24257
I20260808 18:22:27.626080 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance
I20260808 18:22:27.626448 23842 external_mini_cluster.cc:1428] Setting key c67e8d1ebb408cec01b3abdedd88a3fa
Aug 08 18:22:27 dist-test-slave-5k54 krb5kdc[23852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213347, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.131@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:22:27.638871 23944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:27.627477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:59733 (local address 127.23.72.190:42745)
0808 18:22:27.627622 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:27.627628 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:27.628640 (+ 1012us) server_negotiation.cc:407] Connection header received
0808 18:22:27.629915 (+ 1275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:27.629918 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:27.629965 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:27.630034 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:27.631205 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:27.631862 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:27.632965 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:27.633179 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:27.633262 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:27.635672 (+ 2410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:27.635675 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:27.635677 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:27.635702 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:27.637056 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:27.637539 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:27.637542 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:27.637543 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:27.637632 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:27.637940 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:27.637942 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:27.637944 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:27.638125 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:27.638202 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:27.638641 (+ 439us) server_negotiation.cc:299] Negotiation successful
0808 18:22:27.638741 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 18:22:27.639760 24400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:27.627955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:42745 (local address 127.23.72.131:59733)
0808 18:22:27.628477 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:27.628529 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:27.629712 (+ 1183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:27.630193 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:27.630199 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:27.630576 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:27.631040 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:27.631053 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:27.631980 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:27.631983 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:27.632856 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:27.632865 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:27.633306 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:27.633308 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:27.633455 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:27.633976 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:27.633991 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:27.635498 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:27.637161 (+ 1663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:27.637166 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:27.637178 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:27.637435 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:27.637720 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:27.637722 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:27.637724 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:27.637854 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:27.638212 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:27.638217 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:27.638443 (+ 226us) client_negotiation.cc:769] Sending connection context
0808 18:22:27.638595 (+ 152us) client_negotiation.cc:240] Negotiation successful
0808 18:22:27.638824 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":164,"threads_started":1}
I20260808 18:22:27.641043 24398 heartbeater.cc:347] Connected to a master server at 127.23.72.190:42745
I20260808 18:22:27.641283 24398 heartbeater.cc:464] Registering TS with master...
I20260808 18:22:27.641810 24398 heartbeater.cc:511] Master 127.23.72.190:42745 requested a full tablet report, sending...
I20260808 18:22:27.642802 23901 ts_manager.cc:194] Registered new tserver with Master: a78f5aaacd4c45ca819be17b54450c79 (127.23.72.131:37233)
I20260808 18:22:27.643376 23901 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.131@KRBTEST.COM'} at 127.23.72.131:59733
I20260808 18:22:27.653098 23842 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 18:22:27 dist-test-slave-5k54 krb5kdc[23852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213347, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 18:22:27 dist-test-slave-5k54 krb5kdc[23852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213347, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.23.72.129@KRBTEST.COM
I20260808 18:22:27.680466 24405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:27.670664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51626 (local address 127.23.72.129:37789)
0808 18:22:27.670938 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:27.670956 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:27.670989 (+ 33us) server_negotiation.cc:407] Connection header received
0808 18:22:27.671084 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:27.671089 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:27.671281 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:27.671411 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:27.672129 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:27.672897 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:27.673676 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:27.674285 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:27.674382 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:27.676930 (+ 2548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:27.676940 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:27.676952 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:27.676975 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:27.678573 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:27.679124 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:27.679129 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:27.679130 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:27.679210 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:27.679511 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:27.679513 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:27.679515 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:27.679743 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:27.679897 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:27.680200 (+ 303us) server_negotiation.cc:299] Negotiation successful
0808 18:22:27.680324 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":99,"threads_started":1}
W20260808 18:22:27.681411 24017 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51626
Aug 08 18:22:27 dist-test-slave-5k54 krb5kdc[23852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213347, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 18:22:27 dist-test-slave-5k54 krb5kdc[23852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213347, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.72.129@KRBTEST.COM
I20260808 18:22:27.708161 24405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:27.699938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51640 (local address 127.23.72.129:37789)
0808 18:22:27.700124 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:27.700128 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:27.700195 (+ 67us) server_negotiation.cc:407] Connection header received
0808 18:22:27.700300 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:27.700303 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:27.700359 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:27.700427 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:27.701081 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:27.701580 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:27.702375 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:27.702590 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:27.702668 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:27.705243 (+ 2575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:27.705246 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:27.705248 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:27.705271 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:27.706542 (+ 1271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:27.706893 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:27.706895 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:27.706897 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:27.706965 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:27.707181 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:27.707183 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:27.707184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:27.707346 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:27.707525 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:27.707883 (+ 358us) server_negotiation.cc:299] Negotiation successful
0808 18:22:27.708039 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 18:22:27.709173 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 23972
I20260808 18:22:27.716882 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 24113
I20260808 18:22:27.723104 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 24257
I20260808 18:22:27.729174 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 23865
2026-08-08T18:22:27Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2292 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:22:27 dist-test-slave-5k54 krb5kdc[24412](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:22:27 dist-test-slave-5k54 krb5kdc[24412](info): set up 2 sockets
Aug 08 18:22:27 dist-test-slave-5k54 krb5kdc[24412](info): commencing operation
krb5kdc: starting...
W20260808 18:22:28.812693 23842 mini_kdc.cc:121] Time spent starting KDC: real 1.059s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Aug 08 18:22:28 dist-test-slave-5k54 krb5kdc[24412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:22:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:22:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.72.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
I20260808 18:22:28.947510 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:38683
--webserver_interface=127.23.72.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:44655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:38683
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:29.064265 24427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:29.064522 24427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:29.064559 24427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:29.068728 24427 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:22:29.068790 24427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:29.068816 24427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:29.068833 24427 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:22:29.068889 24427 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:22:29.074024 24427 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:44655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:38683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:38683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.72.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:29.075071 24427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:29.076162 24427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 18:22:29.084975 24427 server_base.cc:1034] running on GCE node
W20260808 18:22:29.085104 24436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:29.085064 24434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:29.085064 24433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:29.085893 24427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:29.086983 24427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:29.088169 24427 hybrid_clock.cc:648] HybridClock initialized: now 1786213349088155 us; error 27 us; skew 500 ppm
Aug 08 18:22:29 dist-test-slave-5k54 krb5kdc[24412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213349, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:29.094234 24427 init.cc:377] Logged in from keytab as kudu/127.23.72.190@KRBTEST.COM (short username kudu)
I20260808 18:22:29.095130 24427 webserver.cc:460] Webserver started at http://127.23.72.190:33373/ using document root <none> and password file <none>
I20260808 18:22:29.095752 24427 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:29.095844 24427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:29.096093 24427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:29.098141 24427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "4cb1e9f68ee94515ba6ae8c2fb252628"
format_stamp: "Formatted at 2026-08-08 18:22:29 on dist-test-slave-5k54"
server_key: "1f0a36522e8d999300fa5229a1c77029"
server_key_iv: "ff4addf06bdc105b6b4418051aff8aa2"
server_key_version: "encryptionkey@0"
I20260808 18:22:29.098716 24427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "4cb1e9f68ee94515ba6ae8c2fb252628"
format_stamp: "Formatted at 2026-08-08 18:22:29 on dist-test-slave-5k54"
server_key: "1f0a36522e8d999300fa5229a1c77029"
server_key_iv: "ff4addf06bdc105b6b4418051aff8aa2"
server_key_version: "encryptionkey@0"
I20260808 18:22:29.102435 24427 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260808 18:22:29.105044 24443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:29.106218 24427 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:22:29.106356 24427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "4cb1e9f68ee94515ba6ae8c2fb252628"
format_stamp: "Formatted at 2026-08-08 18:22:29 on dist-test-slave-5k54"
server_key: "1f0a36522e8d999300fa5229a1c77029"
server_key_iv: "ff4addf06bdc105b6b4418051aff8aa2"
server_key_version: "encryptionkey@0"
I20260808 18:22:29.106470 24427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:29.143703 24427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:29.146781 24427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:29.147001 24427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:29.154466 24427 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.190:38683
I20260808 18:22:29.154497 24498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.190:38683 every 8 connection(s)
I20260808 18:22:29.155443 24427 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:22:29.158427 24499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:29.164474 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 24427
I20260808 18:22:29.164595 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:22:29.164608 24499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cb1e9f68ee94515ba6ae8c2fb252628: Bootstrap starting.
I20260808 18:22:29.164882 23842 external_mini_cluster.cc:1428] Setting key 35201c7804a7b3b92ad078038bed5a03
I20260808 18:22:29.167259 24499 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4cb1e9f68ee94515ba6ae8c2fb252628: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:29.168052 24499 log.cc:824] T 00000000000000000000000000000000 P 4cb1e9f68ee94515ba6ae8c2fb252628: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:29.169943 24499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cb1e9f68ee94515ba6ae8c2fb252628: No bootstrap required, opened a new log
Aug 08 18:22:29 dist-test-slave-5k54 krb5kdc[24412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:22:29.174564 24499 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4cb1e9f68ee94515ba6ae8c2fb252628 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cb1e9f68ee94515ba6ae8c2fb252628" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 38683 } }
I20260808 18:22:29.174820 24499 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4cb1e9f68ee94515ba6ae8c2fb252628 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:29.174902 24499 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4cb1e9f68ee94515ba6ae8c2fb252628 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cb1e9f68ee94515ba6ae8c2fb252628, State: Initialized, Role: FOLLOWER
I20260808 18:22:29.175490 24499 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4cb1e9f68ee94515ba6ae8c2fb252628 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cb1e9f68ee94515ba6ae8c2fb252628" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 38683 } }
I20260808 18:22:29.175659 24499 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4cb1e9f68ee94515ba6ae8c2fb252628 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:22:29.175770 24499 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4cb1e9f68ee94515ba6ae8c2fb252628 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:22:29.175901 24499 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4cb1e9f68ee94515ba6ae8c2fb252628 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:29.176821 24499 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4cb1e9f68ee94515ba6ae8c2fb252628 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cb1e9f68ee94515ba6ae8c2fb252628" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 38683 } }
I20260808 18:22:29.177237 24499 leader_election.cc:302] T 00000000000000000000000000000000 P 4cb1e9f68ee94515ba6ae8c2fb252628 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4cb1e9f68ee94515ba6ae8c2fb252628; no voters:
I20260808 18:22:29.177597 24499 leader_election.cc:288] T 00000000000000000000000000000000 P 4cb1e9f68ee94515ba6ae8c2fb252628 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:22:29.177743 24504 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4cb1e9f68ee94515ba6ae8c2fb252628 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:22:29.178043 24504 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4cb1e9f68ee94515ba6ae8c2fb252628 [term 1 LEADER]: Becoming Leader. State: Replica: 4cb1e9f68ee94515ba6ae8c2fb252628, State: Running, Role: LEADER
I20260808 18:22:29.178403 24504 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4cb1e9f68ee94515ba6ae8c2fb252628 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cb1e9f68ee94515ba6ae8c2fb252628" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 38683 } }
I20260808 18:22:29.178921 24499 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4cb1e9f68ee94515ba6ae8c2fb252628 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:22:29.179658 24506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cb1e9f68ee94515ba6ae8c2fb252628 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4cb1e9f68ee94515ba6ae8c2fb252628. Latest consensus state: current_term: 1 leader_uuid: "4cb1e9f68ee94515ba6ae8c2fb252628" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cb1e9f68ee94515ba6ae8c2fb252628" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 38683 } } }
I20260808 18:22:29.179877 24506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cb1e9f68ee94515ba6ae8c2fb252628 [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:29.180159 24505 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cb1e9f68ee94515ba6ae8c2fb252628 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4cb1e9f68ee94515ba6ae8c2fb252628" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cb1e9f68ee94515ba6ae8c2fb252628" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 38683 } } }
I20260808 18:22:29.180311 24505 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cb1e9f68ee94515ba6ae8c2fb252628 [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:29.180394 24513 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:22:29.180696 24502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:29.166340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44746 (local address 127.23.72.190:38683)
0808 18:22:29.166762 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:29.166777 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:29.166813 (+ 36us) server_negotiation.cc:407] Connection header received
0808 18:22:29.167564 (+ 751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:29.167582 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:29.167906 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:29.168234 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:29.169034 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:29.169995 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:29.171220 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:29.171767 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:29.171888 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:29.174865 (+ 2977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:29.174871 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:29.174889 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:29.174921 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:29.176905 (+ 1984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:29.177453 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:29.177458 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:29.177464 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:29.177624 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:29.177974 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:29.177979 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:29.177980 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:29.178530 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:29.178719 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:29.179173 (+ 454us) server_negotiation.cc:299] Negotiation successful
0808 18:22:29.179529 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":121,"threads_started":1}
I20260808 18:22:29.183666 24513 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:22:29.189369 24513 catalog_manager.cc:1347] Generated new cluster ID: 2fb8668a70e44bbda2c85e020d2509b8
I20260808 18:22:29.189486 24513 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:22:29.200537 24513 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:22:29.201455 24513 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:22:29.210412 24513 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4cb1e9f68ee94515ba6ae8c2fb252628: Generated new TSK 0
I20260808 18:22:29.211184 24513 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.72.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab.
Entry for principal kudu/127.23.72.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab.
I20260808 18:22:29.251250 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.129:0
--local_ip_for_outbound_sockets=127.23.72.129
--webserver_interface=127.23.72.129
--webserver_port=0
--tserver_master_addrs=127.23.72.190:38683
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab
--principal=kudu/127.23.72.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:44655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:29.364228 24526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:29.364504 24526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:29.364579 24526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:29.368714 24526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:29.368796 24526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:29.368891 24526 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.129
I20260808 18:22:29.373979 24526 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:44655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab
--principal=kudu/127.23.72.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.72.129
--webserver_port=0
--tserver_master_addrs=127.23.72.190:38683
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.129
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:29.374993 24526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:29.376070 24526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:29.385771 24532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:29.385809 24533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:29.385833 24536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:29.385987 24526 server_base.cc:1034] running on GCE node
I20260808 18:22:29.386469 24526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:29.387076 24526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:29.388263 24526 hybrid_clock.cc:648] HybridClock initialized: now 1786213349388212 us; error 67 us; skew 500 ppm
Aug 08 18:22:29 dist-test-slave-5k54 krb5kdc[24412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213349, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:29.391096 24526 init.cc:377] Logged in from keytab as kudu/127.23.72.129@KRBTEST.COM (short username kudu)
I20260808 18:22:29.392016 24526 webserver.cc:460] Webserver started at http://127.23.72.129:33981/ using document root <none> and password file <none>
I20260808 18:22:29.392561 24526 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:29.392649 24526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:29.392879 24526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:29.394855 24526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-0/data/instance:
uuid: "9e77e9b1f3dd4d70b60c047f9d17a6e0"
format_stamp: "Formatted at 2026-08-08 18:22:29 on dist-test-slave-5k54"
server_key: "ad6a66377bfe5abd4cbecbb5b43b4a67"
server_key_iv: "8c8a3d0a9064ab02a031c010fb1c44d6"
server_key_version: "encryptionkey@0"
I20260808 18:22:29.395406 24526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance:
uuid: "9e77e9b1f3dd4d70b60c047f9d17a6e0"
format_stamp: "Formatted at 2026-08-08 18:22:29 on dist-test-slave-5k54"
server_key: "ad6a66377bfe5abd4cbecbb5b43b4a67"
server_key_iv: "8c8a3d0a9064ab02a031c010fb1c44d6"
server_key_version: "encryptionkey@0"
I20260808 18:22:29.400028 24526 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 18:22:29.402472 24543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:29.403781 24526 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:22:29.403915 24526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-0/wal
uuid: "9e77e9b1f3dd4d70b60c047f9d17a6e0"
format_stamp: "Formatted at 2026-08-08 18:22:29 on dist-test-slave-5k54"
server_key: "ad6a66377bfe5abd4cbecbb5b43b4a67"
server_key_iv: "8c8a3d0a9064ab02a031c010fb1c44d6"
server_key_version: "encryptionkey@0"
I20260808 18:22:29.404031 24526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:29.419185 24526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:29.422614 24526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:29.422830 24526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:29.423475 24526 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:22:29.424317 24526 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:22:29.424388 24526 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:29.424455 24526 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:22:29.424501 24526 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:29.434500 24526 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.129:39425
I20260808 18:22:29.434520 24665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.129:39425 every 8 connection(s)
I20260808 18:22:29.435475 24526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
I20260808 18:22:29.438380 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 24526
I20260808 18:22:29.438499 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance
I20260808 18:22:29.438794 23842 external_mini_cluster.cc:1428] Setting key 87404c1d51d470976694e19f9e11604d
Aug 08 18:22:29 dist-test-slave-5k54 krb5kdc[24412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213349, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.72.130@KRBTEST.COM; defaulting to no policy
I20260808 18:22:29.450630 24502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:29.437311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:45751 (local address 127.23.72.190:38683)
0808 18:22:29.437463 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:29.437467 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:29.438672 (+ 1205us) server_negotiation.cc:407] Connection header received
0808 18:22:29.439845 (+ 1173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:29.439849 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:29.439894 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:29.439980 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:29.441677 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:29.442193 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:29.443505 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:29.443752 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:29.443874 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:29.446876 (+ 3002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:29.446879 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:29.446897 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:29.446925 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:29.448418 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:29.448994 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:29.448997 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:29.448999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:29.449092 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:29.449491 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:29.449494 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:29.449496 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:29.449695 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:29.449810 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:29.450321 (+ 511us) server_negotiation.cc:299] Negotiation successful
0808 18:22:29.450459 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 18:22:29.451568 24668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:29.437631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:38683 (local address 127.23.72.129:45751)
0808 18:22:29.438492 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:29.438555 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:29.439644 (+ 1089us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:29.440182 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:29.440191 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:29.440719 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:29.441509 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:29.441532 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:29.442326 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:29.442332 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:29.443346 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:29.443356 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:29.443918 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:29.443923 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:29.444096 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:29.444733 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:29.444750 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:29.446687 (+ 1937us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:29.448576 (+ 1889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:29.448582 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:29.448610 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:29.448877 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:29.449187 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:29.449191 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:29.449193 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:29.449390 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:29.449807 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:29.449815 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:29.450079 (+ 264us) client_negotiation.cc:769] Sending connection context
0808 18:22:29.450279 (+ 200us) client_negotiation.cc:240] Negotiation successful
0808 18:22:29.450628 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":119,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.130@KRBTEST.COM" created.
I20260808 18:22:29.453061 24666 heartbeater.cc:347] Connected to a master server at 127.23.72.190:38683
I20260808 18:22:29.453373 24666 heartbeater.cc:464] Registering TS with master...
I20260808 18:22:29.453969 24666 heartbeater.cc:511] Master 127.23.72.190:38683 requested a full tablet report, sending...
I20260808 18:22:29.455632 24461 ts_manager.cc:194] Registered new tserver with Master: 9e77e9b1f3dd4d70b60c047f9d17a6e0 (127.23.72.129:39425)
I20260808 18:22:29.457293 24461 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.129@KRBTEST.COM'} at 127.23.72.129:45751
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab.
Entry for principal kudu/127.23.72.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab.
I20260808 18:22:29.470213 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.130:0
--local_ip_for_outbound_sockets=127.23.72.130
--webserver_interface=127.23.72.130
--webserver_port=0
--tserver_master_addrs=127.23.72.190:38683
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab
--principal=kudu/127.23.72.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:44655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:29.580520 24672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:29.580760 24672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:29.580801 24672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:29.584682 24672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:29.584730 24672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:29.584800 24672 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.130
I20260808 18:22:29.589781 24672 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:44655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab
--principal=kudu/127.23.72.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.72.130
--webserver_port=0
--tserver_master_addrs=127.23.72.190:38683
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24672
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.130
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:29.590677 24672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:29.591657 24672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:29.600831 24681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:29.600872 24677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:29.600872 24679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:29.601032 24672 server_base.cc:1034] running on GCE node
I20260808 18:22:29.601816 24672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:29.602458 24672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:29.603642 24672 hybrid_clock.cc:648] HybridClock initialized: now 1786213349603616 us; error 41 us; skew 500 ppm
Aug 08 18:22:29 dist-test-slave-5k54 krb5kdc[24412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213349, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:29.606376 24672 init.cc:377] Logged in from keytab as kudu/127.23.72.130@KRBTEST.COM (short username kudu)
I20260808 18:22:29.607259 24672 webserver.cc:460] Webserver started at http://127.23.72.130:39643/ using document root <none> and password file <none>
I20260808 18:22:29.607851 24672 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:29.607941 24672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:29.608206 24672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:29.610231 24672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-1/data/instance:
uuid: "ac7ba43d44d84f17a327a86d42112d53"
format_stamp: "Formatted at 2026-08-08 18:22:29 on dist-test-slave-5k54"
server_key: "554dc58011e4dc36e304ce88db17e10e"
server_key_iv: "7aab6d7d9819176c45418732582b488c"
server_key_version: "encryptionkey@0"
I20260808 18:22:29.610781 24672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance:
uuid: "ac7ba43d44d84f17a327a86d42112d53"
format_stamp: "Formatted at 2026-08-08 18:22:29 on dist-test-slave-5k54"
server_key: "554dc58011e4dc36e304ce88db17e10e"
server_key_iv: "7aab6d7d9819176c45418732582b488c"
server_key_version: "encryptionkey@0"
I20260808 18:22:29.614401 24672 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 18:22:29.616824 24689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:29.618000 24672 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:22:29.618139 24672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-1/wal
uuid: "ac7ba43d44d84f17a327a86d42112d53"
format_stamp: "Formatted at 2026-08-08 18:22:29 on dist-test-slave-5k54"
server_key: "554dc58011e4dc36e304ce88db17e10e"
server_key_iv: "7aab6d7d9819176c45418732582b488c"
server_key_version: "encryptionkey@0"
I20260808 18:22:29.618255 24672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:29.641145 24672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:29.644330 24672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:29.644560 24672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:29.645165 24672 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:22:29.646109 24672 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:22:29.646180 24672 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:29.646247 24672 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:22:29.646294 24672 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:29.655385 24672 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.130:32965
I20260808 18:22:29.655409 24810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.130:32965 every 8 connection(s)
I20260808 18:22:29.656275 24672 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
I20260808 18:22:29.656759 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 24672
I20260808 18:22:29.656894 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance
I20260808 18:22:29.657208 23842 external_mini_cluster.cc:1428] Setting key 7f67efaa3bcef61cc92ee4a2f13dcb24
Aug 08 18:22:29 dist-test-slave-5k54 krb5kdc[24412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213349, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.130@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.72.131@KRBTEST.COM; defaulting to no policy
I20260808 18:22:29.671784 24502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:29.658375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:36013 (local address 127.23.72.190:38683)
0808 18:22:29.658531 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:29.658536 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:29.659678 (+ 1142us) server_negotiation.cc:407] Connection header received
0808 18:22:29.660609 (+ 931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:29.660612 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:29.660668 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:29.660766 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:29.662126 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:29.662640 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:29.664478 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:29.664818 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:29.664906 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:29.667993 (+ 3087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:29.667996 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:29.667998 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:29.668023 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:29.669579 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:29.670134 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:29.670137 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:29.670138 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:29.670214 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:29.670565 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:29.670568 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:29.670570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:29.670752 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:29.670875 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:29.671410 (+ 535us) server_negotiation.cc:299] Negotiation successful
0808 18:22:29.671585 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.131@KRBTEST.COM" created.
I20260808 18:22:29.672662 24813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:29.658779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:38683 (local address 127.23.72.130:36013)
0808 18:22:29.659489 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:29.659576 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:29.660396 (+ 820us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:29.660941 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:29.660952 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:29.661309 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:29.661960 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:29.661982 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:29.662780 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:29.662786 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:29.664325 (+ 1539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:29.664341 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:29.664983 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:29.664991 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:29.665211 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:29.665876 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:29.665902 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:29.667812 (+ 1910us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:29.669701 (+ 1889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:29.669707 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:29.669721 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:29.669990 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:29.670315 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:29.670319 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:29.670321 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:29.670460 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:29.670870 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:29.670877 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:29.671123 (+ 246us) client_negotiation.cc:769] Sending connection context
0808 18:22:29.671315 (+ 192us) client_negotiation.cc:240] Negotiation successful
0808 18:22:29.671671 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":327,"threads_started":1}
I20260808 18:22:29.673879 24811 heartbeater.cc:347] Connected to a master server at 127.23.72.190:38683
I20260808 18:22:29.674175 24811 heartbeater.cc:464] Registering TS with master...
I20260808 18:22:29.674789 24811 heartbeater.cc:511] Master 127.23.72.190:38683 requested a full tablet report, sending...
I20260808 18:22:29.676308 24461 ts_manager.cc:194] Registered new tserver with Master: ac7ba43d44d84f17a327a86d42112d53 (127.23.72.130:32965)
I20260808 18:22:29.677259 24461 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.130@KRBTEST.COM'} at 127.23.72.130:36013
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab.
Entry for principal kudu/127.23.72.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab.
I20260808 18:22:29.689942 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.131:0
--local_ip_for_outbound_sockets=127.23.72.131
--webserver_interface=127.23.72.131
--webserver_port=0
--tserver_master_addrs=127.23.72.190:38683
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab
--principal=kudu/127.23.72.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:44655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:29.804318 24816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:29.804565 24816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:29.804608 24816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:29.808729 24816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:29.808786 24816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:29.808846 24816 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.131
I20260808 18:22:29.814136 24816 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:44655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab
--principal=kudu/127.23.72.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.72.131
--webserver_port=0
--tserver_master_addrs=127.23.72.190:38683
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24816
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.131
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:29.815088 24816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:29.816113 24816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:29.826022 24824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:29.825997 24822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:29.825997 24821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:29.826462 24816 server_base.cc:1034] running on GCE node
I20260808 18:22:29.827044 24816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:29.827801 24816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:29.828997 24816 hybrid_clock.cc:648] HybridClock initialized: now 1786213349828961 us; error 51 us; skew 500 ppm
Aug 08 18:22:29 dist-test-slave-5k54 krb5kdc[24412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213349, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:29.831915 24816 init.cc:377] Logged in from keytab as kudu/127.23.72.131@KRBTEST.COM (short username kudu)
I20260808 18:22:29.832769 24816 webserver.cc:460] Webserver started at http://127.23.72.131:34047/ using document root <none> and password file <none>
I20260808 18:22:29.833348 24816 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:29.833453 24816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:29.833691 24816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:29.835997 24816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-2/data/instance:
uuid: "17c25612261f42e1abaf6c4cfe4f78c7"
format_stamp: "Formatted at 2026-08-08 18:22:29 on dist-test-slave-5k54"
server_key: "92446aa72723f2c799962afd7c64d2df"
server_key_iv: "26c71c9a2262f603725327bfe8e0bca3"
server_key_version: "encryptionkey@0"
I20260808 18:22:29.836587 24816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance:
uuid: "17c25612261f42e1abaf6c4cfe4f78c7"
format_stamp: "Formatted at 2026-08-08 18:22:29 on dist-test-slave-5k54"
server_key: "92446aa72723f2c799962afd7c64d2df"
server_key_iv: "26c71c9a2262f603725327bfe8e0bca3"
server_key_version: "encryptionkey@0"
I20260808 18:22:29.840471 24816 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 18:22:29.842938 24831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:29.844208 24816 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:22:29.844342 24816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-2/wal
uuid: "17c25612261f42e1abaf6c4cfe4f78c7"
format_stamp: "Formatted at 2026-08-08 18:22:29 on dist-test-slave-5k54"
server_key: "92446aa72723f2c799962afd7c64d2df"
server_key_iv: "26c71c9a2262f603725327bfe8e0bca3"
server_key_version: "encryptionkey@0"
I20260808 18:22:29.844462 24816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:29.856227 24816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:29.859378 24816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:29.859623 24816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:29.860239 24816 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:22:29.861089 24816 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:22:29.861137 24816 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:29.861207 24816 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:22:29.861243 24816 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:29.870772 24816 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.131:37797
I20260808 18:22:29.870795 24950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.131:37797 every 8 connection(s)
I20260808 18:22:29.871768 24816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
I20260808 18:22:29.876710 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 24816
I20260808 18:22:29.876857 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance
I20260808 18:22:29.877140 23842 external_mini_cluster.cc:1428] Setting key b86e408d0d09d8edb3bc00d7564ef8f5
Aug 08 18:22:29 dist-test-slave-5k54 krb5kdc[24412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213349, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.131@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:22:29.885862 24502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:29.873597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:45369 (local address 127.23.72.190:38683)
0808 18:22:29.873737 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:29.873741 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:29.874543 (+ 802us) server_negotiation.cc:407] Connection header received
0808 18:22:29.875474 (+ 931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:29.875477 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:29.875537 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:29.875640 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:29.876786 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:29.877487 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:29.879003 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:29.879368 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:29.879542 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:29.882259 (+ 2717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:29.882262 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:29.882265 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:29.882295 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:29.883784 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:29.884325 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:29.884331 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:29.884332 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:29.884423 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:29.884772 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:29.884777 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:29.884779 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:29.884968 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:29.885071 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:29.885557 (+ 486us) server_negotiation.cc:299] Negotiation successful
0808 18:22:29.885684 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 18:22:29.886631 24953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:29.873946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:38683 (local address 127.23.72.131:45369)
0808 18:22:29.874381 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:29.874436 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:29.875236 (+ 800us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:29.875814 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:29.875820 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:29.876174 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:29.876639 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:29.876652 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:29.877647 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:29.877651 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:29.878883 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:29.878893 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:29.879578 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:29.879582 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:29.879804 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:29.880438 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:29.880453 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:29.882094 (+ 1641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:29.883943 (+ 1849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:29.883949 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:29.883963 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:29.884218 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:29.884526 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:29.884529 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:29.884531 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:29.884664 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:29.885084 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:29.885092 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:29.885323 (+ 231us) client_negotiation.cc:769] Sending connection context
0808 18:22:29.885516 (+ 193us) client_negotiation.cc:240] Negotiation successful
0808 18:22:29.885837 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":112,"threads_started":1}
I20260808 18:22:29.887841 24951 heartbeater.cc:347] Connected to a master server at 127.23.72.190:38683
I20260808 18:22:29.888095 24951 heartbeater.cc:464] Registering TS with master...
I20260808 18:22:29.888574 24951 heartbeater.cc:511] Master 127.23.72.190:38683 requested a full tablet report, sending...
I20260808 18:22:29.889612 24461 ts_manager.cc:194] Registered new tserver with Master: 17c25612261f42e1abaf6c4cfe4f78c7 (127.23.72.131:37797)
I20260808 18:22:29.890314 24461 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.131@KRBTEST.COM'} at 127.23.72.131:45369
I20260808 18:22:29.891234 23842 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:22:29.902073 24502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:29.894710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44752 (local address 127.23.72.190:38683)
0808 18:22:29.894844 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:29.894847 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:29.894949 (+ 102us) server_negotiation.cc:407] Connection header received
0808 18:22:29.895043 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:29.895045 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:29.895077 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:29.895142 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:29.895915 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:29.896396 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:29.897185 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:29.897403 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:29.897491 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:29.898757 (+ 1266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:29.898760 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:29.898762 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:29.898785 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:29.900438 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:29.900854 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:29.900856 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:29.900857 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:29.900936 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:29.901181 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:29.901183 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:29.901184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:29.901399 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:29.901504 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:29.901740 (+ 236us) server_negotiation.cc:299] Negotiation successful
0808 18:22:29.901879 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 18:22:29.907030 24461 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44752:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:22:29.910336 24461 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:22:29.924994 24967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:29.919635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35794 (local address 127.23.72.131:37797)
0808 18:22:29.920079 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:29.920093 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:29.920117 (+ 24us) server_negotiation.cc:407] Connection header received
0808 18:22:29.920199 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:29.920203 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:29.920343 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:29.920505 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:22:29.921002 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:29.921838 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:29.923598 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:29.924355 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:29.924478 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:29.924645 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:29.924703 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 18:22:29.924830 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":130,"threads_started":1}
I20260808 18:22:29.924978 24966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:29.919201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.131:37797 (local address 127.0.0.1:35794)
0808 18:22:29.919657 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:29.919670 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:29.919749 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:29.920768 (+ 1019us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:29.920769 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:29.920783 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:22:29.920897 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:29.920902 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:29.922002 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:29.922007 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:29.923480 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:29.923491 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:29.924482 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:29.924485 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:29.924562 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:29.924592 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 18:22:29.924669 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 18:22:29.924755 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"mutex_wait_us":108,"thread_start_us":87,"threads_started":1}
I20260808 18:22:29.925534 24964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:29.918404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.130:32965 (local address 127.0.0.1:51044)
0808 18:22:29.919183 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:29.919387 (+ 204us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:29.919967 (+ 580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:29.920670 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:29.920672 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:29.920687 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:22:29.920833 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:29.920840 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:29.922242 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:29.922245 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:29.923754 (+ 1509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:29.923765 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:29.924882 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:29.924886 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:29.924974 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:29.925001 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 18:22:29.925087 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 18:22:29.925396 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"mutex_wait_us":80,"thread_start_us":81,"threads_started":1}
I20260808 18:22:29.925988 24963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:29.918522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51044 (local address 127.23.72.130:32965)
0808 18:22:29.918854 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:29.918868 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:29.919504 (+ 636us) server_negotiation.cc:407] Connection header received
0808 18:22:29.920212 (+ 708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:29.920217 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:29.920361 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:29.920489 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:22:29.920976 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:29.922111 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:29.923884 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:29.924735 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:29.924858 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:29.924996 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:29.925734 (+ 738us) server_negotiation.cc:299] Negotiation successful
0808 18:22:29.925841 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":111,"threads_started":1}
I20260808 18:22:29.927970 24881 tablet_service.cc:1467] Processing CreateTablet for tablet af6f51bca77140a58fe1d04899dea651 (DEFAULT_TABLE table=test-table [id=bb02f6000ad74cb6832e89cd508ac9c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:22:29.928414 24741 tablet_service.cc:1467] Processing CreateTablet for tablet af6f51bca77140a58fe1d04899dea651 (DEFAULT_TABLE table=test-table [id=bb02f6000ad74cb6832e89cd508ac9c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:22:29.928966 24881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af6f51bca77140a58fe1d04899dea651. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:29.929478 24741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af6f51bca77140a58fe1d04899dea651. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:29.929720 24962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:29.918545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:39425 (local address 127.0.0.1:54320)
0808 18:22:29.919205 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:29.919244 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:29.919383 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:29.920498 (+ 1115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:29.920503 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:29.920525 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:22:29.920661 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:29.920674 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:29.922190 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:29.922196 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:29.923761 (+ 1565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:29.923766 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:29.929352 (+ 5586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:29.929357 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:29.929451 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:29.929476 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 18:22:29.929529 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 18:22:29.929597 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"mutex_wait_us":176,"thread_start_us":101,"threads_started":1}
I20260808 18:22:29.930433 24965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:29.919080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54320 (local address 127.23.72.129:39425)
0808 18:22:29.919982 (+ 902us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:29.919999 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:29.920020 (+ 21us) server_negotiation.cc:407] Connection header received
0808 18:22:29.920096 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:29.920100 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:29.920323 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:29.920489 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:22:29.920785 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:29.922030 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:29.925062 (+ 3032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:29.929188 (+ 4126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:29.929997 (+ 809us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:29.930138 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:29.930190 (+ 52us) server_negotiation.cc:299] Negotiation successful
0808 18:22:29.930279 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":793,"thread_start_us":119,"threads_started":1}
I20260808 18:22:29.933240 24593 tablet_service.cc:1467] Processing CreateTablet for tablet af6f51bca77140a58fe1d04899dea651 (DEFAULT_TABLE table=test-table [id=bb02f6000ad74cb6832e89cd508ac9c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:22:29.934259 24593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af6f51bca77140a58fe1d04899dea651. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:29.935513 24968 tablet_bootstrap.cc:492] T af6f51bca77140a58fe1d04899dea651 P 17c25612261f42e1abaf6c4cfe4f78c7: Bootstrap starting.
I20260808 18:22:29.937180 24969 tablet_bootstrap.cc:492] T af6f51bca77140a58fe1d04899dea651 P ac7ba43d44d84f17a327a86d42112d53: Bootstrap starting.
I20260808 18:22:29.937855 24968 tablet_bootstrap.cc:654] T af6f51bca77140a58fe1d04899dea651 P 17c25612261f42e1abaf6c4cfe4f78c7: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:29.938905 24968 log.cc:824] T af6f51bca77140a58fe1d04899dea651 P 17c25612261f42e1abaf6c4cfe4f78c7: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:29.939657 24969 tablet_bootstrap.cc:654] T af6f51bca77140a58fe1d04899dea651 P ac7ba43d44d84f17a327a86d42112d53: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:29.940483 24969 log.cc:824] T af6f51bca77140a58fe1d04899dea651 P ac7ba43d44d84f17a327a86d42112d53: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:29.941229 24972 tablet_bootstrap.cc:492] T af6f51bca77140a58fe1d04899dea651 P 9e77e9b1f3dd4d70b60c047f9d17a6e0: Bootstrap starting.
I20260808 18:22:29.941255 24968 tablet_bootstrap.cc:492] T af6f51bca77140a58fe1d04899dea651 P 17c25612261f42e1abaf6c4cfe4f78c7: No bootstrap required, opened a new log
I20260808 18:22:29.941442 24968 ts_tablet_manager.cc:1397] T af6f51bca77140a58fe1d04899dea651 P 17c25612261f42e1abaf6c4cfe4f78c7: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 18:22:29.942468 24969 tablet_bootstrap.cc:492] T af6f51bca77140a58fe1d04899dea651 P ac7ba43d44d84f17a327a86d42112d53: No bootstrap required, opened a new log
I20260808 18:22:29.942642 24969 ts_tablet_manager.cc:1397] T af6f51bca77140a58fe1d04899dea651 P ac7ba43d44d84f17a327a86d42112d53: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260808 18:22:29.943079 24972 tablet_bootstrap.cc:654] T af6f51bca77140a58fe1d04899dea651 P 9e77e9b1f3dd4d70b60c047f9d17a6e0: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:29.944144 24972 log.cc:824] T af6f51bca77140a58fe1d04899dea651 P 9e77e9b1f3dd4d70b60c047f9d17a6e0: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:29.945693 24972 tablet_bootstrap.cc:492] T af6f51bca77140a58fe1d04899dea651 P 9e77e9b1f3dd4d70b60c047f9d17a6e0: No bootstrap required, opened a new log
I20260808 18:22:29.945878 24972 ts_tablet_manager.cc:1397] T af6f51bca77140a58fe1d04899dea651 P 9e77e9b1f3dd4d70b60c047f9d17a6e0: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 18:22:29.945708 24968 raft_consensus.cc:348] T af6f51bca77140a58fe1d04899dea651 P 17c25612261f42e1abaf6c4cfe4f78c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac7ba43d44d84f17a327a86d42112d53" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 32965 } } peers { permanent_uuid: "9e77e9b1f3dd4d70b60c047f9d17a6e0" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 39425 } } peers { permanent_uuid: "17c25612261f42e1abaf6c4cfe4f78c7" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 37797 } }
I20260808 18:22:29.946091 24968 raft_consensus.cc:374] T af6f51bca77140a58fe1d04899dea651 P 17c25612261f42e1abaf6c4cfe4f78c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:29.946188 24968 raft_consensus.cc:729] T af6f51bca77140a58fe1d04899dea651 P 17c25612261f42e1abaf6c4cfe4f78c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17c25612261f42e1abaf6c4cfe4f78c7, State: Initialized, Role: FOLLOWER
I20260808 18:22:29.946453 24969 raft_consensus.cc:348] T af6f51bca77140a58fe1d04899dea651 P ac7ba43d44d84f17a327a86d42112d53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac7ba43d44d84f17a327a86d42112d53" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 32965 } } peers { permanent_uuid: "9e77e9b1f3dd4d70b60c047f9d17a6e0" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 39425 } } peers { permanent_uuid: "17c25612261f42e1abaf6c4cfe4f78c7" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 37797 } }
I20260808 18:22:29.946753 24969 raft_consensus.cc:374] T af6f51bca77140a58fe1d04899dea651 P ac7ba43d44d84f17a327a86d42112d53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:29.946847 24969 raft_consensus.cc:729] T af6f51bca77140a58fe1d04899dea651 P ac7ba43d44d84f17a327a86d42112d53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac7ba43d44d84f17a327a86d42112d53, State: Initialized, Role: FOLLOWER
I20260808 18:22:29.946859 24968 consensus_queue.cc:260] T af6f51bca77140a58fe1d04899dea651 P 17c25612261f42e1abaf6c4cfe4f78c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac7ba43d44d84f17a327a86d42112d53" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 32965 } } peers { permanent_uuid: "9e77e9b1f3dd4d70b60c047f9d17a6e0" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 39425 } } peers { permanent_uuid: "17c25612261f42e1abaf6c4cfe4f78c7" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 37797 } }
I20260808 18:22:29.947336 24969 consensus_queue.cc:260] T af6f51bca77140a58fe1d04899dea651 P ac7ba43d44d84f17a327a86d42112d53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac7ba43d44d84f17a327a86d42112d53" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 32965 } } peers { permanent_uuid: "9e77e9b1f3dd4d70b60c047f9d17a6e0" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 39425 } } peers { permanent_uuid: "17c25612261f42e1abaf6c4cfe4f78c7" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 37797 } }
I20260808 18:22:29.947625 24951 heartbeater.cc:503] Master 127.23.72.190:38683 was elected leader, sending a full tablet report...
I20260808 18:22:29.948063 24968 ts_tablet_manager.cc:1428] T af6f51bca77140a58fe1d04899dea651 P 17c25612261f42e1abaf6c4cfe4f78c7: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 18:22:29.948432 24969 ts_tablet_manager.cc:1428] T af6f51bca77140a58fe1d04899dea651 P ac7ba43d44d84f17a327a86d42112d53: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 18:22:29.948617 24811 heartbeater.cc:503] Master 127.23.72.190:38683 was elected leader, sending a full tablet report...
I20260808 18:22:29.949935 24972 raft_consensus.cc:348] T af6f51bca77140a58fe1d04899dea651 P 9e77e9b1f3dd4d70b60c047f9d17a6e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac7ba43d44d84f17a327a86d42112d53" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 32965 } } peers { permanent_uuid: "9e77e9b1f3dd4d70b60c047f9d17a6e0" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 39425 } } peers { permanent_uuid: "17c25612261f42e1abaf6c4cfe4f78c7" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 37797 } }
I20260808 18:22:29.950171 24972 raft_consensus.cc:374] T af6f51bca77140a58fe1d04899dea651 P 9e77e9b1f3dd4d70b60c047f9d17a6e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:29.950223 24972 raft_consensus.cc:729] T af6f51bca77140a58fe1d04899dea651 P 9e77e9b1f3dd4d70b60c047f9d17a6e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e77e9b1f3dd4d70b60c047f9d17a6e0, State: Initialized, Role: FOLLOWER
I20260808 18:22:29.950691 24972 consensus_queue.cc:260] T af6f51bca77140a58fe1d04899dea651 P 9e77e9b1f3dd4d70b60c047f9d17a6e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac7ba43d44d84f17a327a86d42112d53" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 32965 } } peers { permanent_uuid: "9e77e9b1f3dd4d70b60c047f9d17a6e0" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 39425 } } peers { permanent_uuid: "17c25612261f42e1abaf6c4cfe4f78c7" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 37797 } }
I20260808 18:22:29.951560 24666 heartbeater.cc:503] Master 127.23.72.190:38683 was elected leader, sending a full tablet report...
I20260808 18:22:29.951674 24972 ts_tablet_manager.cc:1428] T af6f51bca77140a58fe1d04899dea651 P 9e77e9b1f3dd4d70b60c047f9d17a6e0: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260808 18:22:30.122838 24952 tablet.cc:2369] T af6f51bca77140a58fe1d04899dea651 P 17c25612261f42e1abaf6c4cfe4f78c7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 18:22:30.157464 24812 tablet.cc:2369] T af6f51bca77140a58fe1d04899dea651 P ac7ba43d44d84f17a327a86d42112d53: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:22:30.159709 24974 raft_consensus.cc:482] T af6f51bca77140a58fe1d04899dea651 P 17c25612261f42e1abaf6c4cfe4f78c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:22:30.159952 24974 raft_consensus.cc:504] T af6f51bca77140a58fe1d04899dea651 P 17c25612261f42e1abaf6c4cfe4f78c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac7ba43d44d84f17a327a86d42112d53" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 32965 } } peers { permanent_uuid: "9e77e9b1f3dd4d70b60c047f9d17a6e0" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 39425 } } peers { permanent_uuid: "17c25612261f42e1abaf6c4cfe4f78c7" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 37797 } }
I20260808 18:22:30.161073 24974 leader_election.cc:288] T af6f51bca77140a58fe1d04899dea651 P 17c25612261f42e1abaf6c4cfe4f78c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ac7ba43d44d84f17a327a86d42112d53 (127.23.72.130:32965), 9e77e9b1f3dd4d70b60c047f9d17a6e0 (127.23.72.129:39425)
I20260808 18:22:30.165496 24965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:30.161525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:60313 (local address 127.23.72.129:39425)
0808 18:22:30.161663 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:30.161670 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:30.162205 (+ 535us) server_negotiation.cc:407] Connection header received
0808 18:22:30.162314 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:30.162317 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:30.162368 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:30.162449 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:22:30.162720 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:30.163322 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:30.164456 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:30.164875 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:30.165007 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:30.165064 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:30.165267 (+ 203us) server_negotiation.cc:299] Negotiation successful
0808 18:22:30.165350 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 18:22:30.165619 24953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:30.161321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.130:32965 (local address 127.23.72.131:56953)
0808 18:22:30.161473 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:30.161491 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:30.161591 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:30.161913 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:30.161916 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:30.161939 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:22:30.162057 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:30.162063 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:30.163051 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:30.163054 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:30.164326 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:30.164337 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:30.165208 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:30.165213 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:30.165334 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:30.165352 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 18:22:30.165426 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 18:22:30.165490 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"mutex_wait_us":8}
I20260808 18:22:30.165650 24963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:30.161540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:56953 (local address 127.23.72.130:32965)
0808 18:22:30.161656 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:30.161660 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:30.161675 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:22:30.161725 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:30.161728 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:30.161785 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:30.161867 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:22:30.162185 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:30.162938 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:30.164472 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:30.165088 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:30.165205 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:30.165250 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:30.165456 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 18:22:30.165523 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 18:22:30.165935 24977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:30.161333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:39425 (local address 127.23.72.131:60313)
0808 18:22:30.162126 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:30.162140 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:30.162230 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:30.162475 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:30.162478 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:30.162494 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:22:30.162603 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:30.162608 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:30.163458 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:30.163461 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:30.164322 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:30.164332 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:30.165001 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:30.165005 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:30.165109 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:30.165132 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 18:22:30.165694 (+ 562us) client_negotiation.cc:240] Negotiation successful
0808 18:22:30.165768 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":728,"mutex_wait_us":378,"thread_start_us":145,"threads_started":1}
I20260808 18:22:30.166244 24761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af6f51bca77140a58fe1d04899dea651" candidate_uuid: "17c25612261f42e1abaf6c4cfe4f78c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac7ba43d44d84f17a327a86d42112d53" is_pre_election: true
I20260808 18:22:30.166503 24616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af6f51bca77140a58fe1d04899dea651" candidate_uuid: "17c25612261f42e1abaf6c4cfe4f78c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e77e9b1f3dd4d70b60c047f9d17a6e0" is_pre_election: true
I20260808 18:22:30.166564 24761 raft_consensus.cc:2454] T af6f51bca77140a58fe1d04899dea651 P ac7ba43d44d84f17a327a86d42112d53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17c25612261f42e1abaf6c4cfe4f78c7 in term 0.
I20260808 18:22:30.166783 24616 raft_consensus.cc:2454] T af6f51bca77140a58fe1d04899dea651 P 9e77e9b1f3dd4d70b60c047f9d17a6e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17c25612261f42e1abaf6c4cfe4f78c7 in term 0.
I20260808 18:22:30.167163 24836 leader_election.cc:302] T af6f51bca77140a58fe1d04899dea651 P 17c25612261f42e1abaf6c4cfe4f78c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17c25612261f42e1abaf6c4cfe4f78c7, ac7ba43d44d84f17a327a86d42112d53; no voters:
I20260808 18:22:30.167508 24974 raft_consensus.cc:2781] T af6f51bca77140a58fe1d04899dea651 P 17c25612261f42e1abaf6c4cfe4f78c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:22:30.167598 24974 raft_consensus.cc:482] T af6f51bca77140a58fe1d04899dea651 P 17c25612261f42e1abaf6c4cfe4f78c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:22:30.167640 24974 raft_consensus.cc:3037] T af6f51bca77140a58fe1d04899dea651 P 17c25612261f42e1abaf6c4cfe4f78c7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:30.168839 24974 raft_consensus.cc:504] T af6f51bca77140a58fe1d04899dea651 P 17c25612261f42e1abaf6c4cfe4f78c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac7ba43d44d84f17a327a86d42112d53" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 32965 } } peers { permanent_uuid: "9e77e9b1f3dd4d70b60c047f9d17a6e0" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 39425 } } peers { permanent_uuid: "17c25612261f42e1abaf6c4cfe4f78c7" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 37797 } }
I20260808 18:22:30.169245 24974 leader_election.cc:288] T af6f51bca77140a58fe1d04899dea651 P 17c25612261f42e1abaf6c4cfe4f78c7 [CANDIDATE]: Term 1 election: Requested vote from peers ac7ba43d44d84f17a327a86d42112d53 (127.23.72.130:32965), 9e77e9b1f3dd4d70b60c047f9d17a6e0 (127.23.72.129:39425)
I20260808 18:22:30.169612 24761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af6f51bca77140a58fe1d04899dea651" candidate_uuid: "17c25612261f42e1abaf6c4cfe4f78c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac7ba43d44d84f17a327a86d42112d53"
I20260808 18:22:30.169687 24616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af6f51bca77140a58fe1d04899dea651" candidate_uuid: "17c25612261f42e1abaf6c4cfe4f78c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e77e9b1f3dd4d70b60c047f9d17a6e0"
I20260808 18:22:30.169762 24761 raft_consensus.cc:3037] T af6f51bca77140a58fe1d04899dea651 P ac7ba43d44d84f17a327a86d42112d53 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:30.169858 24616 raft_consensus.cc:3037] T af6f51bca77140a58fe1d04899dea651 P 9e77e9b1f3dd4d70b60c047f9d17a6e0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:30.170863 24761 raft_consensus.cc:2454] T af6f51bca77140a58fe1d04899dea651 P ac7ba43d44d84f17a327a86d42112d53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17c25612261f42e1abaf6c4cfe4f78c7 in term 1.
I20260808 18:22:30.170917 24616 raft_consensus.cc:2454] T af6f51bca77140a58fe1d04899dea651 P 9e77e9b1f3dd4d70b60c047f9d17a6e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17c25612261f42e1abaf6c4cfe4f78c7 in term 1.
I20260808 18:22:30.171186 24836 leader_election.cc:302] T af6f51bca77140a58fe1d04899dea651 P 17c25612261f42e1abaf6c4cfe4f78c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17c25612261f42e1abaf6c4cfe4f78c7, ac7ba43d44d84f17a327a86d42112d53; no voters:
I20260808 18:22:30.171343 24974 raft_consensus.cc:2781] T af6f51bca77140a58fe1d04899dea651 P 17c25612261f42e1abaf6c4cfe4f78c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:22:30.171633 24974 raft_consensus.cc:686] T af6f51bca77140a58fe1d04899dea651 P 17c25612261f42e1abaf6c4cfe4f78c7 [term 1 LEADER]: Becoming Leader. State: Replica: 17c25612261f42e1abaf6c4cfe4f78c7, State: Running, Role: LEADER
I20260808 18:22:30.171979 24974 consensus_queue.cc:237] T af6f51bca77140a58fe1d04899dea651 P 17c25612261f42e1abaf6c4cfe4f78c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac7ba43d44d84f17a327a86d42112d53" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 32965 } } peers { permanent_uuid: "9e77e9b1f3dd4d70b60c047f9d17a6e0" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 39425 } } peers { permanent_uuid: "17c25612261f42e1abaf6c4cfe4f78c7" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 37797 } }
I20260808 18:22:30.175318 24461 catalog_manager.cc:5495] T af6f51bca77140a58fe1d04899dea651 P 17c25612261f42e1abaf6c4cfe4f78c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 17c25612261f42e1abaf6c4cfe4f78c7 (127.23.72.131). New cstate: current_term: 1 leader_uuid: "17c25612261f42e1abaf6c4cfe4f78c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac7ba43d44d84f17a327a86d42112d53" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 32965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e77e9b1f3dd4d70b60c047f9d17a6e0" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 39425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17c25612261f42e1abaf6c4cfe4f78c7" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 37797 } health_report { overall_health: HEALTHY } } }
W20260808 18:22:30.186906 24667 tablet.cc:2369] T af6f51bca77140a58fe1d04899dea651 P 9e77e9b1f3dd4d70b60c047f9d17a6e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 18:22:30 dist-test-slave-5k54 krb5kdc[24412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.72.129@KRBTEST.COM
I20260808 18:22:30.218462 24965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:30.208670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54332 (local address 127.23.72.129:39425)
0808 18:22:30.208879 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:30.208885 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:30.209069 (+ 184us) server_negotiation.cc:407] Connection header received
0808 18:22:30.209252 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:30.209255 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:30.209315 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:30.209407 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:30.210196 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:30.210776 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:30.211770 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:30.212050 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:30.212172 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:30.214888 (+ 2716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:30.214904 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:30.214935 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:30.214961 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:30.216695 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:30.217180 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:30.217187 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:30.217188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:30.217305 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:30.217636 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:30.217639 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:30.217640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:30.217897 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:30.218063 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:30.218230 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 18:22:30.218329 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
Aug 08 18:22:30 dist-test-slave-5k54 krb5kdc[24412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213350, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 18:22:30 dist-test-slave-5k54 krb5kdc[24412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213350, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.23.72.129@KRBTEST.COM
I20260808 18:22:30.247253 24965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:30.238287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54344 (local address 127.23.72.129:39425)
0808 18:22:30.238595 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:30.238599 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:30.238615 (+ 16us) server_negotiation.cc:407] Connection header received
0808 18:22:30.238660 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:30.238662 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:30.238703 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:30.238775 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:30.239488 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:30.240034 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:30.240912 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:30.241154 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:30.241248 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:30.243714 (+ 2466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:30.243718 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:30.243720 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:30.243745 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:30.245440 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:30.245898 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:30.245902 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:30.245903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:30.245984 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:30.246230 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:30.246233 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:30.246234 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:30.246399 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:30.246499 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:30.246925 (+ 426us) server_negotiation.cc:299] Negotiation successful
0808 18:22:30.247061 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197}
Aug 08 18:22:30 dist-test-slave-5k54 krb5kdc[24412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213350, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 18:22:30 dist-test-slave-5k54 krb5kdc[24412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213350, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.72.129@KRBTEST.COM
I20260808 18:22:30.273979 24965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:30.265670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54352 (local address 127.23.72.129:39425)
0808 18:22:30.265867 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:30.265871 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:30.265946 (+ 75us) server_negotiation.cc:407] Connection header received
0808 18:22:30.266057 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:30.266060 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:30.266105 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:30.266178 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:30.266812 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:30.267324 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:30.268213 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:30.268428 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:30.268508 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:30.270947 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:30.270949 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:30.270952 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:30.270974 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:30.272457 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:30.272894 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:30.272897 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:30.272898 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:30.272969 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:30.273193 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:30.273195 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:30.273196 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:30.273346 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:30.273439 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:30.273642 (+ 203us) server_negotiation.cc:299] Negotiation successful
0808 18:22:30.273735 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260808 18:22:30.275355 24572 tablet_service.cc:2908] Rejecting scan request for tablet af6f51bca77140a58fe1d04899dea651: Uninitialized: clean time has not yet been initialized
I20260808 18:22:30.277628 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 24526
I20260808 18:22:30.285458 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 24672
I20260808 18:22:30.292176 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 24816
I20260808 18:22:30.298947 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 24427
2026-08-08T18:22:30Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2569 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:22:30 dist-test-slave-5k54 krb5kdc[24995](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:22:30 dist-test-slave-5k54 krb5kdc[24995](info): set up 2 sockets
Aug 08 18:22:30 dist-test-slave-5k54 krb5kdc[24995](info): commencing operation
krb5kdc: starting...
W20260808 18:22:31.388909 23842 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Aug 08 18:22:31 dist-test-slave-5k54 krb5kdc[24995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:22:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:22:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.72.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
I20260808 18:22:31.524920 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:40501
--webserver_interface=127.23.72.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:46459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:40501
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:31.635766 25008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:31.636071 25008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:31.636145 25008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:31.640120 25008 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:22:31.640209 25008 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 18:22:31.640240 25008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:31.640285 25008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:31.640322 25008 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:22:31.640352 25008 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:22:31.645879 25008 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:46459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:40501
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:40501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.72.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25008
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:31.646952 25008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:31.648008 25008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:31.656787 25013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:31.656773 25014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:31.656755 25016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:31.657028 25008 server_base.cc:1034] running on GCE node
I20260808 18:22:31.657677 25008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:31.658872 25008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:31.660069 25008 hybrid_clock.cc:648] HybridClock initialized: now 1786213351660045 us; error 48 us; skew 500 ppm
Aug 08 18:22:31 dist-test-slave-5k54 krb5kdc[24995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213351, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:31.663100 25008 init.cc:377] Logged in from keytab as kudu/127.23.72.190@KRBTEST.COM (short username kudu)
I20260808 18:22:31.664036 25008 webserver.cc:460] Webserver started at http://127.23.72.190:34615/ using document root <none> and password file <none>
I20260808 18:22:31.664593 25008 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:31.664687 25008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:31.664935 25008 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:31.666935 25008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "01de4188925043e6810d12c91104af7b"
format_stamp: "Formatted at 2026-08-08 18:22:31 on dist-test-slave-5k54"
server_key: "1b8d8f880ab70518ad0e892761732fab"
server_key_iv: "ddce3f31cc7992c1714174347b3dfaf4"
server_key_version: "encryptionkey@0"
I20260808 18:22:31.667558 25008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "01de4188925043e6810d12c91104af7b"
format_stamp: "Formatted at 2026-08-08 18:22:31 on dist-test-slave-5k54"
server_key: "1b8d8f880ab70518ad0e892761732fab"
server_key_iv: "ddce3f31cc7992c1714174347b3dfaf4"
server_key_version: "encryptionkey@0"
I20260808 18:22:31.671375 25008 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 18:22:31.674115 25023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:31.675388 25008 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:22:31.675557 25008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "01de4188925043e6810d12c91104af7b"
format_stamp: "Formatted at 2026-08-08 18:22:31 on dist-test-slave-5k54"
server_key: "1b8d8f880ab70518ad0e892761732fab"
server_key_iv: "ddce3f31cc7992c1714174347b3dfaf4"
server_key_version: "encryptionkey@0"
I20260808 18:22:31.675678 25008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:31.694455 25008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:31.698020 25008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:31.698239 25008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:31.707321 25008 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.190:40501
I20260808 18:22:31.707413 25090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.190:40501 every 8 connection(s)
I20260808 18:22:31.708325 25008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:22:31.711100 25091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:31.711889 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 25008
I20260808 18:22:31.712007 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:22:31.712286 23842 external_mini_cluster.cc:1428] Setting key 31a7a5a2209d2f328724a30d4b590581
I20260808 18:22:31.718010 25091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01de4188925043e6810d12c91104af7b: Bootstrap starting.
I20260808 18:22:31.720535 25091 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01de4188925043e6810d12c91104af7b: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:31.721328 25091 log.cc:824] T 00000000000000000000000000000000 P 01de4188925043e6810d12c91104af7b: Log is configured to *not* fsync() on all Append() calls
Aug 08 18:22:31 dist-test-slave-5k54 krb5kdc[24995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:22:31.723497 25091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01de4188925043e6810d12c91104af7b: No bootstrap required, opened a new log
I20260808 18:22:31.726473 25091 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01de4188925043e6810d12c91104af7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01de4188925043e6810d12c91104af7b" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40501 } }
I20260808 18:22:31.726769 25091 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01de4188925043e6810d12c91104af7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:31.726845 25091 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01de4188925043e6810d12c91104af7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01de4188925043e6810d12c91104af7b, State: Initialized, Role: FOLLOWER
I20260808 18:22:31.727288 25091 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01de4188925043e6810d12c91104af7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01de4188925043e6810d12c91104af7b" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40501 } }
I20260808 18:22:31.727471 25091 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01de4188925043e6810d12c91104af7b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:22:31.727558 25091 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01de4188925043e6810d12c91104af7b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:22:31.727684 25091 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01de4188925043e6810d12c91104af7b [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:31.727849 25094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:31.713635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47408 (local address 127.23.72.190:40501)
0808 18:22:31.714035 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:31.714055 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:31.714098 (+ 43us) server_negotiation.cc:407] Connection header received
0808 18:22:31.714951 (+ 853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:31.714974 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:31.715391 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:31.715777 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:31.716414 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:31.717931 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:31.718850 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:31.719577 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:31.719732 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:31.722438 (+ 2706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:31.722443 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:31.722457 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:31.722483 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:31.724827 (+ 2344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:31.725290 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:31.725296 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:31.725301 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:31.725422 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:31.725711 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:31.725715 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:31.725716 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:31.726097 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:31.726311 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:31.726567 (+ 256us) server_negotiation.cc:299] Negotiation successful
0808 18:22:31.726892 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":99,"threads_started":1}
I20260808 18:22:31.728611 25091 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01de4188925043e6810d12c91104af7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01de4188925043e6810d12c91104af7b" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40501 } }
I20260808 18:22:31.728958 25091 leader_election.cc:302] T 00000000000000000000000000000000 P 01de4188925043e6810d12c91104af7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01de4188925043e6810d12c91104af7b; no voters:
I20260808 18:22:31.729262 25091 leader_election.cc:288] T 00000000000000000000000000000000 P 01de4188925043e6810d12c91104af7b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:22:31.729382 25096 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01de4188925043e6810d12c91104af7b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:22:31.729573 25096 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01de4188925043e6810d12c91104af7b [term 1 LEADER]: Becoming Leader. State: Replica: 01de4188925043e6810d12c91104af7b, State: Running, Role: LEADER
I20260808 18:22:31.729923 25096 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01de4188925043e6810d12c91104af7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01de4188925043e6810d12c91104af7b" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40501 } }
I20260808 18:22:31.730522 25091 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01de4188925043e6810d12c91104af7b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:22:31.731101 25096 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01de4188925043e6810d12c91104af7b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01de4188925043e6810d12c91104af7b. Latest consensus state: current_term: 1 leader_uuid: "01de4188925043e6810d12c91104af7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01de4188925043e6810d12c91104af7b" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40501 } } }
I20260808 18:22:31.731256 25096 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01de4188925043e6810d12c91104af7b [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:31.731738 25103 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:22:31.731769 25097 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01de4188925043e6810d12c91104af7b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "01de4188925043e6810d12c91104af7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01de4188925043e6810d12c91104af7b" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40501 } } }
I20260808 18:22:31.731952 25097 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01de4188925043e6810d12c91104af7b [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:31.734825 25103 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:22:31.740097 25103 catalog_manager.cc:1347] Generated new cluster ID: 3f62abb62d25486f8b8bea4893d007d4
I20260808 18:22:31.740170 25103 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:22:31.746577 25103 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:22:31.747502 25103 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:22:31.751274 25103 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01de4188925043e6810d12c91104af7b: Generated new TSK 0
I20260808 18:22:31.751935 25103 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.72.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab.
Entry for principal kudu/127.23.72.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab.
I20260808 18:22:31.787118 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.129:0
--local_ip_for_outbound_sockets=127.23.72.129
--webserver_interface=127.23.72.129
--webserver_port=0
--tserver_master_addrs=127.23.72.190:40501
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab
--principal=kudu/127.23.72.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:46459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:31.899662 25118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:31.899942 25118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:31.900018 25118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:31.904013 25118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:31.904088 25118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:31.904157 25118 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 18:22:31.904234 25118 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.129
I20260808 18:22:31.909329 25118 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:46459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab
--principal=kudu/127.23.72.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.72.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.72.190:40501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25118
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.129
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:31.910334 25118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:31.911337 25118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:31.920794 25124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:31.920845 25126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:31.920818 25123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:31.921170 25118 server_base.cc:1034] running on GCE node
I20260808 18:22:31.921599 25118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:31.922199 25118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:31.923372 25118 hybrid_clock.cc:648] HybridClock initialized: now 1786213351923335 us; error 32 us; skew 500 ppm
Aug 08 18:22:31 dist-test-slave-5k54 krb5kdc[24995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213351, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:31.926254 25118 init.cc:377] Logged in from keytab as kudu/127.23.72.129@KRBTEST.COM (short username kudu)
I20260808 18:22:31.927075 25118 webserver.cc:460] Webserver started at http://127.23.72.129:42111/ using document root <none> and password file <none>
I20260808 18:22:31.927645 25118 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:31.927708 25118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:31.927875 25118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:31.929821 25118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-0/data/instance:
uuid: "5553dc2bdc4248aca8e7b3e808a7a95e"
format_stamp: "Formatted at 2026-08-08 18:22:31 on dist-test-slave-5k54"
server_key: "761a40d25e8811bc32eab4d186c0c72e"
server_key_iv: "c29879627d0f872b07ef17b6aa660057"
server_key_version: "encryptionkey@0"
I20260808 18:22:31.930332 25118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance:
uuid: "5553dc2bdc4248aca8e7b3e808a7a95e"
format_stamp: "Formatted at 2026-08-08 18:22:31 on dist-test-slave-5k54"
server_key: "761a40d25e8811bc32eab4d186c0c72e"
server_key_iv: "c29879627d0f872b07ef17b6aa660057"
server_key_version: "encryptionkey@0"
I20260808 18:22:31.934128 25118 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 18:22:31.936492 25133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:31.937686 25118 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:22:31.937844 25118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-0/wal
uuid: "5553dc2bdc4248aca8e7b3e808a7a95e"
format_stamp: "Formatted at 2026-08-08 18:22:31 on dist-test-slave-5k54"
server_key: "761a40d25e8811bc32eab4d186c0c72e"
server_key_iv: "c29879627d0f872b07ef17b6aa660057"
server_key_version: "encryptionkey@0"
I20260808 18:22:31.937943 25118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:31.952026 25118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:31.954891 25118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:31.955088 25118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:31.956521 25118 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:22:31.956597 25118 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:31.956657 25118 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:22:31.956705 25118 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 18:22:31 dist-test-slave-5k54 krb5kdc[24995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213351, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:22:31.967993 25118 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.129:41447
I20260808 18:22:31.968169 25255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.129:41447 every 8 connection(s)
I20260808 18:22:31.968879 25118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
I20260808 18:22:31.970674 25094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:31.957492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:40431 (local address 127.23.72.190:40501)
0808 18:22:31.957618 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:31.957621 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:31.958624 (+ 1003us) server_negotiation.cc:407] Connection header received
0808 18:22:31.959789 (+ 1165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:31.959792 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:31.959849 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:31.959955 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:31.961340 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:31.961923 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:31.963179 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:31.963593 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:31.963712 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:31.966812 (+ 3100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:31.966815 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:31.966817 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:31.966841 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:31.968555 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:31.969114 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:31.969118 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:31.969120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:31.969214 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:31.969586 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:31.969590 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:31.969592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:31.969836 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:31.969949 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:31.970454 (+ 505us) server_negotiation.cc:299] Negotiation successful
0808 18:22:31.970557 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 18:22:31.971477 25143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:31.958018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:40501 (local address 127.23.72.129:40431)
0808 18:22:31.958488 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:31.958528 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:31.959570 (+ 1042us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:31.960115 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:31.960122 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:31.960683 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:31.961199 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:31.961214 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:31.962065 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:31.962068 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:31.963050 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:31.963059 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:31.963761 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:31.963765 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:31.963986 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:31.964587 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:31.964614 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:31.966630 (+ 2016us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:31.968679 (+ 2049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:31.968685 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:31.968698 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:31.968998 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:31.969315 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:31.969318 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:31.969320 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:31.969487 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:31.969948 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:31.969953 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:31.970230 (+ 277us) client_negotiation.cc:769] Sending connection context
0808 18:22:31.970394 (+ 164us) client_negotiation.cc:240] Negotiation successful
0808 18:22:31.970621 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":136,"threads_started":1}
I20260808 18:22:31.972767 25256 heartbeater.cc:347] Connected to a master server at 127.23.72.190:40501
I20260808 18:22:31.972988 25256 heartbeater.cc:464] Registering TS with master...
I20260808 18:22:31.973552 25256 heartbeater.cc:511] Master 127.23.72.190:40501 requested a full tablet report, sending...
I20260808 18:22:31.973842 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 25118
I20260808 18:22:31.974004 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance
I20260808 18:22:31.974293 23842 external_mini_cluster.cc:1428] Setting key 5c306af874a23b9618c09efbaceaed04
I20260808 18:22:31.975129 25041 ts_manager.cc:194] Registered new tserver with Master: 5553dc2bdc4248aca8e7b3e808a7a95e (127.23.72.129:41447)
I20260808 18:22:31.976565 25041 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.129@KRBTEST.COM'} at 127.23.72.129:40431
I20260808 18:22:31.984547 25262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:31.975981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:40501 (local address 127.23.72.129:33197)
0808 18:22:31.976301 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:31.976320 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:31.976469 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:31.976830 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:31.976836 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:31.977057 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:31.977208 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:31.977219 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:31.978331 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:31.978336 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:31.979318 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:31.979333 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:31.980054 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:31.980057 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:31.980167 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:31.980670 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:31.980688 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:31.980994 (+ 306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:31.983105 (+ 2111us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:31.983108 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:31.983109 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:31.983301 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:31.983620 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:31.983622 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:31.983623 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:31.983698 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:31.984144 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:31.984146 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:31.984214 (+ 68us) client_negotiation.cc:769] Sending connection context
0808 18:22:31.984292 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 18:22:31.984409 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":111,"threads_started":1}
I20260808 18:22:31.984683 25094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:31.975867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:33197 (local address 127.23.72.190:40501)
0808 18:22:31.975994 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:31.975998 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:31.976397 (+ 399us) server_negotiation.cc:407] Connection header received
0808 18:22:31.976570 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:31.976575 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:31.976627 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:31.976681 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:31.977361 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:31.978156 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:31.979495 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:31.979931 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:31.980047 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:31.981142 (+ 1095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:31.981149 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:31.981152 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:31.981182 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:31.982986 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:31.983402 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:31.983405 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:31.983407 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:31.983519 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:31.983797 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:31.983807 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:31.983809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:31.984035 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:31.984155 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:31.984389 (+ 234us) server_negotiation.cc:299] Negotiation successful
0808 18:22:31.984513 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
WARNING: no policy specified for kudu/127.23.72.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab.
Entry for principal kudu/127.23.72.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab.
I20260808 18:22:32.007247 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.130:0
--local_ip_for_outbound_sockets=127.23.72.130
--webserver_interface=127.23.72.130
--webserver_port=0
--tserver_master_addrs=127.23.72.190:40501
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab
--principal=kudu/127.23.72.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:46459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:32.125263 25265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:32.125531 25265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:32.125576 25265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:32.129753 25265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:32.129815 25265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:32.129854 25265 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 18:22:32.129925 25265 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.130
I20260808 18:22:32.135269 25265 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:46459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab
--principal=kudu/127.23.72.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.72.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.72.190:40501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25265
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.130
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:32.136363 25265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:32.137387 25265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:32.146888 25274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:32.146862 25272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:32.146924 25271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:32.147234 25265 server_base.cc:1034] running on GCE node
I20260808 18:22:32.147704 25265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:32.148387 25265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:32.149590 25265 hybrid_clock.cc:648] HybridClock initialized: now 1786213352149569 us; error 31 us; skew 500 ppm
Aug 08 18:22:32 dist-test-slave-5k54 krb5kdc[24995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213352, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:32.152865 25265 init.cc:377] Logged in from keytab as kudu/127.23.72.130@KRBTEST.COM (short username kudu)
I20260808 18:22:32.153709 25265 webserver.cc:460] Webserver started at http://127.23.72.130:37081/ using document root <none> and password file <none>
I20260808 18:22:32.154244 25265 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:32.154306 25265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:32.154484 25265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:32.156642 25265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-1/data/instance:
uuid: "a741b74692a642c697c799cdf11d6c27"
format_stamp: "Formatted at 2026-08-08 18:22:32 on dist-test-slave-5k54"
server_key: "8c4ff96da02b5ba3731ea292041bcc9f"
server_key_iv: "c1b022e889714d9b4c8fa16964c4b3f0"
server_key_version: "encryptionkey@0"
I20260808 18:22:32.157188 25265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance:
uuid: "a741b74692a642c697c799cdf11d6c27"
format_stamp: "Formatted at 2026-08-08 18:22:32 on dist-test-slave-5k54"
server_key: "8c4ff96da02b5ba3731ea292041bcc9f"
server_key_iv: "c1b022e889714d9b4c8fa16964c4b3f0"
server_key_version: "encryptionkey@0"
I20260808 18:22:32.161001 25265 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 18:22:32.163503 25281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:32.164789 25265 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:22:32.164958 25265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-1/wal
uuid: "a741b74692a642c697c799cdf11d6c27"
format_stamp: "Formatted at 2026-08-08 18:22:32 on dist-test-slave-5k54"
server_key: "8c4ff96da02b5ba3731ea292041bcc9f"
server_key_iv: "c1b022e889714d9b4c8fa16964c4b3f0"
server_key_version: "encryptionkey@0"
I20260808 18:22:32.165060 25265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:32.183223 25265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:32.186811 25265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:32.187047 25265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:32.188657 25265 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:22:32.188740 25265 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:32.188812 25265 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:22:32.188861 25265 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 18:22:32 dist-test-slave-5k54 krb5kdc[24995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213352, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.130@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:22:32.200716 25265 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.130:36155
I20260808 18:22:32.200749 25403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.130:36155 every 8 connection(s)
I20260808 18:22:32.201936 25265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
I20260808 18:22:32.204255 25094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:32.189553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:60919 (local address 127.23.72.190:40501)
0808 18:22:32.189712 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:32.189717 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:32.190582 (+ 865us) server_negotiation.cc:407] Connection header received
0808 18:22:32.191711 (+ 1129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:32.191715 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:32.191779 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:32.191866 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:32.193400 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.194428 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.196066 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.196531 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.196678 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:32.199694 (+ 3016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:32.199702 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:32.199705 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:32.199738 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:32.201729 (+ 1991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:32.202435 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:32.202440 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:32.202443 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:32.202552 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:32.202973 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:32.202976 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:32.202979 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:32.203183 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:32.203308 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:32.203906 (+ 598us) server_negotiation.cc:299] Negotiation successful
0808 18:22:32.204043 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 18:22:32.204463 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 25265
I20260808 18:22:32.204581 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance
I20260808 18:22:32.204938 23842 external_mini_cluster.cc:1428] Setting key a665d3478a017189593488b82e31e6b5
I20260808 18:22:32.205209 25291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:32.189957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:40501 (local address 127.23.72.130:60919)
0808 18:22:32.190413 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:32.190462 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:32.191483 (+ 1021us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:32.192035 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:32.192044 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:32.192545 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:32.193205 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:32.193227 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.194574 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.194580 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:32.195888 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:32.195899 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.196725 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.196729 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:32.196907 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:32.197979 (+ 1072us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:32.197995 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:32.199514 (+ 1519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:32.201896 (+ 2382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:32.201901 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:32.201913 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:32.202308 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:32.202674 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:32.202678 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:32.202680 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:32.202855 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:32.203310 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:32.203318 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:32.203643 (+ 325us) client_negotiation.cc:769] Sending connection context
0808 18:22:32.203840 (+ 197us) client_negotiation.cc:240] Negotiation successful
0808 18:22:32.204168 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":139,"threads_started":1}
I20260808 18:22:32.207506 25404 heartbeater.cc:347] Connected to a master server at 127.23.72.190:40501
I20260808 18:22:32.207660 25404 heartbeater.cc:464] Registering TS with master...
I20260808 18:22:32.208225 25404 heartbeater.cc:511] Master 127.23.72.190:40501 requested a full tablet report, sending...
I20260808 18:22:32.209285 25041 ts_manager.cc:194] Registered new tserver with Master: a741b74692a642c697c799cdf11d6c27 (127.23.72.130:36155)
I20260808 18:22:32.210124 25041 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.130@KRBTEST.COM'} at 127.23.72.130:60919
WARNING: no policy specified for kudu/127.23.72.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.131@KRBTEST.COM" created.
I20260808 18:22:32.223096 25413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:32.208721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:40501 (local address 127.23.72.130:38745)
0808 18:22:32.215268 (+ 6547us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:32.215281 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:32.215374 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:32.215641 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:32.215643 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:32.215806 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:32.215902 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:32.215907 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.216771 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.216773 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:32.217329 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:32.217337 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.217882 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.217884 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:32.217955 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:32.218535 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:32.218547 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:32.219021 (+ 474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:32.221232 (+ 2211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:32.221234 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:32.221236 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:32.221425 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:32.221715 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:32.221717 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:32.221718 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:32.221792 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:32.222479 (+ 687us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:32.222485 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:32.222603 (+ 118us) client_negotiation.cc:769] Sending connection context
0808 18:22:32.222730 (+ 127us) client_negotiation.cc:240] Negotiation successful
0808 18:22:32.222893 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6472,"thread_start_us":134,"threads_started":1}
I20260808 18:22:32.223488 25094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:32.208828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:38745 (local address 127.23.72.190:40501)
0808 18:22:32.208950 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:32.208954 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:32.215380 (+ 6426us) server_negotiation.cc:407] Connection header received
0808 18:22:32.215485 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:32.215488 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:32.215538 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:32.215610 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:32.216007 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.216626 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.217530 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.217765 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.217862 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:32.219183 (+ 1321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:32.219186 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:32.219188 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:32.219210 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:32.221111 (+ 1901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:32.221525 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:32.221531 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:32.221535 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:32.221629 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:32.221944 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:32.221949 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:32.221952 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:32.222286 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:32.222421 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:32.222870 (+ 449us) server_negotiation.cc:299] Negotiation successful
0808 18:22:32.223083 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab.
Entry for principal kudu/127.23.72.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab.
I20260808 18:22:32.238368 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.131:0
--local_ip_for_outbound_sockets=127.23.72.131
--webserver_interface=127.23.72.131
--webserver_port=0
--tserver_master_addrs=127.23.72.190:40501
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab
--principal=kudu/127.23.72.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:46459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:32.359802 25415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:32.360087 25415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:32.360142 25415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:32.364634 25415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:32.364688 25415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:32.364732 25415 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 18:22:32.364789 25415 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.131
I20260808 18:22:32.370613 25415 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:46459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab
--principal=kudu/127.23.72.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.72.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.72.190:40501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25415
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.131
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:32.371657 25415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:32.372742 25415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:32.382777 25421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:32.382776 25420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:32.382928 25423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:32.383261 25415 server_base.cc:1034] running on GCE node
I20260808 18:22:32.383705 25415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:32.384406 25415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:32.385600 25415 hybrid_clock.cc:648] HybridClock initialized: now 1786213352385567 us; error 45 us; skew 500 ppm
Aug 08 18:22:32 dist-test-slave-5k54 krb5kdc[24995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213352, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:32.388666 25415 init.cc:377] Logged in from keytab as kudu/127.23.72.131@KRBTEST.COM (short username kudu)
I20260808 18:22:32.389580 25415 webserver.cc:460] Webserver started at http://127.23.72.131:35569/ using document root <none> and password file <none>
I20260808 18:22:32.390177 25415 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:32.390271 25415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:32.390501 25415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:32.392593 25415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-2/data/instance:
uuid: "2a7890b8ca554c3192dcfefc94adcc48"
format_stamp: "Formatted at 2026-08-08 18:22:32 on dist-test-slave-5k54"
server_key: "6420953a5b072a3e43358a86d9c08278"
server_key_iv: "f284461c14dfc92620545418ff358cb9"
server_key_version: "encryptionkey@0"
I20260808 18:22:32.393179 25415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance:
uuid: "2a7890b8ca554c3192dcfefc94adcc48"
format_stamp: "Formatted at 2026-08-08 18:22:32 on dist-test-slave-5k54"
server_key: "6420953a5b072a3e43358a86d9c08278"
server_key_iv: "f284461c14dfc92620545418ff358cb9"
server_key_version: "encryptionkey@0"
I20260808 18:22:32.397122 25415 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 18:22:32.399622 25430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:32.400763 25415 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:22:32.400911 25415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-2/wal
uuid: "2a7890b8ca554c3192dcfefc94adcc48"
format_stamp: "Formatted at 2026-08-08 18:22:32 on dist-test-slave-5k54"
server_key: "6420953a5b072a3e43358a86d9c08278"
server_key_iv: "f284461c14dfc92620545418ff358cb9"
server_key_version: "encryptionkey@0"
I20260808 18:22:32.401034 25415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:32.416734 25415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:32.420089 25415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:32.420288 25415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:32.421749 25415 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:22:32.421833 25415 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:32.421892 25415 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:22:32.421929 25415 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 18:22:32 dist-test-slave-5k54 krb5kdc[24995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213352, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.131@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:22:32.432879 25415 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.131:41455
I20260808 18:22:32.433943 25415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
I20260808 18:22:32.434546 25555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.131:41455 every 8 connection(s)
I20260808 18:22:32.435698 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 25415
I20260808 18:22:32.435916 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance
I20260808 18:22:32.436241 23842 external_mini_cluster.cc:1428] Setting key 4e0abf10712d0014691fa0acf3eaa852
I20260808 18:22:32.437949 25094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:32.422428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:38781 (local address 127.23.72.190:40501)
0808 18:22:32.422628 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:32.422635 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:32.424046 (+ 1411us) server_negotiation.cc:407] Connection header received
0808 18:22:32.425469 (+ 1423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:32.425476 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:32.425564 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:32.425645 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:32.427295 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.427929 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.429734 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.429954 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.430109 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:32.433281 (+ 3172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:32.433285 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:32.433287 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:32.433326 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:32.435094 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:32.435998 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:32.436005 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:32.436007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:32.436116 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:32.436565 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:32.436569 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:32.436572 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:32.436833 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:32.436924 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:32.437628 (+ 704us) server_negotiation.cc:299] Negotiation successful
0808 18:22:32.437751 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260808 18:22:32.438755 25440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:32.423309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:40501 (local address 127.23.72.131:38781)
0808 18:22:32.423859 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:32.423921 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:32.425204 (+ 1283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:32.425858 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:32.425869 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:32.426430 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:32.427144 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:32.427170 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.428071 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.428077 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:32.429611 (+ 1534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:32.429621 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.430164 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.430169 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:32.430352 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:32.430978 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:32.431002 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:32.433079 (+ 2077us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:32.435195 (+ 2116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:32.435201 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:32.435214 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:32.435815 (+ 601us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:32.436238 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:32.436241 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:32.436243 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:32.436434 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:32.436957 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:32.436964 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:32.437320 (+ 356us) client_negotiation.cc:769] Sending connection context
0808 18:22:32.437534 (+ 214us) client_negotiation.cc:240] Negotiation successful
0808 18:22:32.437881 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":176,"threads_started":1}
I20260808 18:22:32.439961 25556 heartbeater.cc:347] Connected to a master server at 127.23.72.190:40501
I20260808 18:22:32.440122 25556 heartbeater.cc:464] Registering TS with master...
I20260808 18:22:32.440650 25556 heartbeater.cc:511] Master 127.23.72.190:40501 requested a full tablet report, sending...
I20260808 18:22:32.441641 25041 ts_manager.cc:194] Registered new tserver with Master: 2a7890b8ca554c3192dcfefc94adcc48 (127.23.72.131:41455)
I20260808 18:22:32.442417 25041 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.131@KRBTEST.COM'} at 127.23.72.131:38781
I20260808 18:22:32.448647 25094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:32.442106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:48941 (local address 127.23.72.190:40501)
0808 18:22:32.442215 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:32.442219 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:32.442495 (+ 276us) server_negotiation.cc:407] Connection header received
0808 18:22:32.442626 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:32.442629 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:32.442676 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:32.442761 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:32.443184 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.443730 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.444527 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.444738 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.444854 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:32.445835 (+ 981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:32.445838 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:32.445841 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:32.445870 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:32.447082 (+ 1212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:32.447526 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:32.447529 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:32.447531 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:32.447609 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:32.447879 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:32.447883 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:32.447885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:32.448052 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:32.448156 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:32.448404 (+ 248us) server_negotiation.cc:299] Negotiation successful
0808 18:22:32.448507 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 18:22:32.448660 25563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:32.442029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:40501 (local address 127.23.72.131:48941)
0808 18:22:32.442404 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:32.442417 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:32.442530 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:32.442776 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:32.442779 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:32.442972 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:32.443064 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:32.443069 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.443849 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.443852 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:32.444408 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:32.444416 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.444859 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.444862 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:32.444964 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:32.445392 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:32.445420 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:32.445705 (+ 285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:32.447195 (+ 1490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:32.447198 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:32.447200 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:32.447390 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:32.447700 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:32.447705 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:32.447707 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:32.447790 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:32.448149 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:32.448152 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:32.448246 (+ 94us) client_negotiation.cc:769] Sending connection context
0808 18:22:32.448349 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 18:22:32.448494 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":220,"threads_started":1}
I20260808 18:22:32.450383 23842 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 18:22:32 dist-test-slave-5k54 krb5kdc[24995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213352, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 18:22:32 dist-test-slave-5k54 krb5kdc[24995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213352, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:22:32.477535 25094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:32.468926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47418 (local address 127.23.72.190:40501)
0808 18:22:32.469081 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:32.469084 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:32.469097 (+ 13us) server_negotiation.cc:407] Connection header received
0808 18:22:32.469135 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:32.469137 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:32.469174 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:32.469265 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:32.470082 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.470604 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.471468 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.471689 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.471817 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:32.474500 (+ 2683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:32.474504 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:32.474509 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:32.474539 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:32.475880 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:32.476314 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:32.476319 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:32.476325 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:32.476400 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:32.476681 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:32.476686 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:32.476688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:32.476877 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:32.477024 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:32.477254 (+ 230us) server_negotiation.cc:299] Negotiation successful
0808 18:22:32.477365 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 18:22:32.480599 25041 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47418:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:22:32.483649 25041 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:22:32.498644 25579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:32.492598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:41447 (local address 127.0.0.1:54018)
0808 18:22:32.493407 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:32.493423 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:32.493530 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:32.494554 (+ 1024us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:32.494557 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:32.494569 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:22:32.494782 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:32.494790 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.496253 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.496257 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:32.497219 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:32.497231 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.498304 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.498309 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:32.498405 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:32.498432 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 18:22:32.498469 (+ 37us) client_negotiation.cc:240] Negotiation successful
0808 18:22:32.498520 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"mutex_wait_us":130,"spinlock_wait_cycles":768,"thread_start_us":125,"threads_started":1}
I20260808 18:22:32.498648 25574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:32.492510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.130:36155 (local address 127.0.0.1:47212)
0808 18:22:32.493287 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:32.493349 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:32.493470 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:32.494375 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:32.494381 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:32.494405 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:22:32.494570 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:32.494579 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.495941 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.495945 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:32.497143 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:32.497153 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.498041 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.498045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:32.498160 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:32.498205 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 18:22:32.498350 (+ 145us) client_negotiation.cc:240] Negotiation successful
0808 18:22:32.498478 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"mutex_wait_us":2,"thread_start_us":80,"threads_started":1}
I20260808 18:22:32.498981 25577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:32.492651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47212 (local address 127.23.72.130:36155)
0808 18:22:32.493846 (+ 1195us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:32.493855 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:32.493875 (+ 20us) server_negotiation.cc:407] Connection header received
0808 18:22:32.493975 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:32.493982 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:32.494192 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:32.494353 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:22:32.494717 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.495667 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.497287 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.497906 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.498040 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:32.498221 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:32.498732 (+ 511us) server_negotiation.cc:299] Negotiation successful
0808 18:22:32.498827 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1103,"thread_start_us":85,"threads_started":1}
I20260808 18:22:32.499807 25576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:32.492910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.131:41455 (local address 127.0.0.1:50062)
0808 18:22:32.493719 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:32.493732 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:32.493814 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:32.494533 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:32.494539 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:32.494558 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:22:32.494803 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:32.494813 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.496253 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.496257 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:32.498236 (+ 1979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:32.498247 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.499350 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.499357 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:32.499516 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:32.499539 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 18:22:32.499618 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 18:22:32.499689 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":748,"mutex_wait_us":110,"thread_start_us":102,"threads_started":1}
I20260808 18:22:32.500129 25580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:32.493319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54018 (local address 127.23.72.129:41447)
0808 18:22:32.494095 (+ 776us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:32.494101 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:32.494120 (+ 19us) server_negotiation.cc:407] Connection header received
0808 18:22:32.494184 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:32.494188 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:32.494376 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:32.494571 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:22:32.494930 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.496109 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.497366 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.498159 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.499681 (+ 1522us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:32.499833 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:32.499893 (+ 60us) server_negotiation.cc:299] Negotiation successful
0808 18:22:32.499985 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":659,"thread_start_us":107,"threads_started":1}
I20260808 18:22:32.501050 25581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:32.493741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50062 (local address 127.23.72.131:41455)
0808 18:22:32.494085 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:32.494096 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:32.494115 (+ 19us) server_negotiation.cc:407] Connection header received
0808 18:22:32.494182 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:32.494187 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:32.494357 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:32.494767 (+ 410us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:22:32.494919 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.496127 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.498372 (+ 2245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.499213 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.499328 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:32.499529 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:32.500101 (+ 572us) server_negotiation.cc:299] Negotiation successful
0808 18:22:32.500198 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":127,"threads_started":1}
I20260808 18:22:32.502159 25333 tablet_service.cc:1467] Processing CreateTablet for tablet 1eb6806bfadb4de68152dc8d721ed66b (DEFAULT_TABLE table=test-table [id=93b2fa4ca304476e9ba5556b1251db43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:22:32.503203 25333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1eb6806bfadb4de68152dc8d721ed66b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:32.502815 25186 tablet_service.cc:1467] Processing CreateTablet for tablet 1eb6806bfadb4de68152dc8d721ed66b (DEFAULT_TABLE table=test-table [id=93b2fa4ca304476e9ba5556b1251db43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:22:32.503890 25186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1eb6806bfadb4de68152dc8d721ed66b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:32.503705 25484 tablet_service.cc:1467] Processing CreateTablet for tablet 1eb6806bfadb4de68152dc8d721ed66b (DEFAULT_TABLE table=test-table [id=93b2fa4ca304476e9ba5556b1251db43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:22:32.504688 25484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1eb6806bfadb4de68152dc8d721ed66b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:32.510605 25582 tablet_bootstrap.cc:492] T 1eb6806bfadb4de68152dc8d721ed66b P a741b74692a642c697c799cdf11d6c27: Bootstrap starting.
I20260808 18:22:32.512073 25584 tablet_bootstrap.cc:492] T 1eb6806bfadb4de68152dc8d721ed66b P 2a7890b8ca554c3192dcfefc94adcc48: Bootstrap starting.
I20260808 18:22:32.512986 25582 tablet_bootstrap.cc:654] T 1eb6806bfadb4de68152dc8d721ed66b P a741b74692a642c697c799cdf11d6c27: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:32.513893 25582 log.cc:824] T 1eb6806bfadb4de68152dc8d721ed66b P a741b74692a642c697c799cdf11d6c27: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:32.514236 25584 tablet_bootstrap.cc:654] T 1eb6806bfadb4de68152dc8d721ed66b P 2a7890b8ca554c3192dcfefc94adcc48: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:32.515012 25583 tablet_bootstrap.cc:492] T 1eb6806bfadb4de68152dc8d721ed66b P 5553dc2bdc4248aca8e7b3e808a7a95e: Bootstrap starting.
I20260808 18:22:32.515098 25584 log.cc:824] T 1eb6806bfadb4de68152dc8d721ed66b P 2a7890b8ca554c3192dcfefc94adcc48: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:32.516117 25582 tablet_bootstrap.cc:492] T 1eb6806bfadb4de68152dc8d721ed66b P a741b74692a642c697c799cdf11d6c27: No bootstrap required, opened a new log
I20260808 18:22:32.516323 25582 ts_tablet_manager.cc:1397] T 1eb6806bfadb4de68152dc8d721ed66b P a741b74692a642c697c799cdf11d6c27: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 18:22:32.517052 25583 tablet_bootstrap.cc:654] T 1eb6806bfadb4de68152dc8d721ed66b P 5553dc2bdc4248aca8e7b3e808a7a95e: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:32.517153 25584 tablet_bootstrap.cc:492] T 1eb6806bfadb4de68152dc8d721ed66b P 2a7890b8ca554c3192dcfefc94adcc48: No bootstrap required, opened a new log
I20260808 18:22:32.517378 25584 ts_tablet_manager.cc:1397] T 1eb6806bfadb4de68152dc8d721ed66b P 2a7890b8ca554c3192dcfefc94adcc48: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.002s
I20260808 18:22:32.517891 25583 log.cc:824] T 1eb6806bfadb4de68152dc8d721ed66b P 5553dc2bdc4248aca8e7b3e808a7a95e: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:32.519869 25583 tablet_bootstrap.cc:492] T 1eb6806bfadb4de68152dc8d721ed66b P 5553dc2bdc4248aca8e7b3e808a7a95e: No bootstrap required, opened a new log
I20260808 18:22:32.519752 25582 raft_consensus.cc:348] T 1eb6806bfadb4de68152dc8d721ed66b P a741b74692a642c697c799cdf11d6c27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a741b74692a642c697c799cdf11d6c27" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36155 } } peers { permanent_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41447 } } peers { permanent_uuid: "2a7890b8ca554c3192dcfefc94adcc48" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41455 } }
I20260808 18:22:32.520056 25583 ts_tablet_manager.cc:1397] T 1eb6806bfadb4de68152dc8d721ed66b P 5553dc2bdc4248aca8e7b3e808a7a95e: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 18:22:32.520076 25582 raft_consensus.cc:374] T 1eb6806bfadb4de68152dc8d721ed66b P a741b74692a642c697c799cdf11d6c27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:32.520154 25582 raft_consensus.cc:729] T 1eb6806bfadb4de68152dc8d721ed66b P a741b74692a642c697c799cdf11d6c27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a741b74692a642c697c799cdf11d6c27, State: Initialized, Role: FOLLOWER
I20260808 18:22:32.520638 25582 consensus_queue.cc:260] T 1eb6806bfadb4de68152dc8d721ed66b P a741b74692a642c697c799cdf11d6c27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a741b74692a642c697c799cdf11d6c27" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36155 } } peers { permanent_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41447 } } peers { permanent_uuid: "2a7890b8ca554c3192dcfefc94adcc48" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41455 } }
I20260808 18:22:32.521245 25584 raft_consensus.cc:348] T 1eb6806bfadb4de68152dc8d721ed66b P 2a7890b8ca554c3192dcfefc94adcc48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a741b74692a642c697c799cdf11d6c27" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36155 } } peers { permanent_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41447 } } peers { permanent_uuid: "2a7890b8ca554c3192dcfefc94adcc48" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41455 } }
I20260808 18:22:32.521543 25584 raft_consensus.cc:374] T 1eb6806bfadb4de68152dc8d721ed66b P 2a7890b8ca554c3192dcfefc94adcc48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:32.521613 25584 raft_consensus.cc:729] T 1eb6806bfadb4de68152dc8d721ed66b P 2a7890b8ca554c3192dcfefc94adcc48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a7890b8ca554c3192dcfefc94adcc48, State: Initialized, Role: FOLLOWER
I20260808 18:22:32.521848 25582 ts_tablet_manager.cc:1428] T 1eb6806bfadb4de68152dc8d721ed66b P a741b74692a642c697c799cdf11d6c27: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 18:22:32.522047 25584 consensus_queue.cc:260] T 1eb6806bfadb4de68152dc8d721ed66b P 2a7890b8ca554c3192dcfefc94adcc48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a741b74692a642c697c799cdf11d6c27" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36155 } } peers { permanent_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41447 } } peers { permanent_uuid: "2a7890b8ca554c3192dcfefc94adcc48" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41455 } }
I20260808 18:22:32.522210 25404 heartbeater.cc:503] Master 127.23.72.190:40501 was elected leader, sending a full tablet report...
I20260808 18:22:32.522715 25556 heartbeater.cc:503] Master 127.23.72.190:40501 was elected leader, sending a full tablet report...
I20260808 18:22:32.523025 25584 ts_tablet_manager.cc:1428] T 1eb6806bfadb4de68152dc8d721ed66b P 2a7890b8ca554c3192dcfefc94adcc48: Time spent starting tablet: real 0.006s user 0.002s sys 0.003s
I20260808 18:22:32.524474 25583 raft_consensus.cc:348] T 1eb6806bfadb4de68152dc8d721ed66b P 5553dc2bdc4248aca8e7b3e808a7a95e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a741b74692a642c697c799cdf11d6c27" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36155 } } peers { permanent_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41447 } } peers { permanent_uuid: "2a7890b8ca554c3192dcfefc94adcc48" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41455 } }
I20260808 18:22:32.524696 25583 raft_consensus.cc:374] T 1eb6806bfadb4de68152dc8d721ed66b P 5553dc2bdc4248aca8e7b3e808a7a95e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:32.524798 25583 raft_consensus.cc:729] T 1eb6806bfadb4de68152dc8d721ed66b P 5553dc2bdc4248aca8e7b3e808a7a95e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5553dc2bdc4248aca8e7b3e808a7a95e, State: Initialized, Role: FOLLOWER
I20260808 18:22:32.525199 25583 consensus_queue.cc:260] T 1eb6806bfadb4de68152dc8d721ed66b P 5553dc2bdc4248aca8e7b3e808a7a95e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a741b74692a642c697c799cdf11d6c27" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36155 } } peers { permanent_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41447 } } peers { permanent_uuid: "2a7890b8ca554c3192dcfefc94adcc48" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41455 } }
I20260808 18:22:32.525964 25256 heartbeater.cc:503] Master 127.23.72.190:40501 was elected leader, sending a full tablet report...
I20260808 18:22:32.526146 25583 ts_tablet_manager.cc:1428] T 1eb6806bfadb4de68152dc8d721ed66b P 5553dc2bdc4248aca8e7b3e808a7a95e: Time spent starting tablet: real 0.006s user 0.002s sys 0.003s
I20260808 18:22:32.619982 25590 raft_consensus.cc:482] T 1eb6806bfadb4de68152dc8d721ed66b P 5553dc2bdc4248aca8e7b3e808a7a95e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:22:32.620229 25590 raft_consensus.cc:504] T 1eb6806bfadb4de68152dc8d721ed66b P 5553dc2bdc4248aca8e7b3e808a7a95e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a741b74692a642c697c799cdf11d6c27" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36155 } } peers { permanent_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41447 } } peers { permanent_uuid: "2a7890b8ca554c3192dcfefc94adcc48" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41455 } }
I20260808 18:22:32.621483 25590 leader_election.cc:288] T 1eb6806bfadb4de68152dc8d721ed66b P 5553dc2bdc4248aca8e7b3e808a7a95e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a741b74692a642c697c799cdf11d6c27 (127.23.72.130:36155), 2a7890b8ca554c3192dcfefc94adcc48 (127.23.72.131:41455)
I20260808 18:22:32.625643 25577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:32.621879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:39447 (local address 127.23.72.130:36155)
0808 18:22:32.622012 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:32.622016 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:32.622329 (+ 313us) server_negotiation.cc:407] Connection header received
0808 18:22:32.622528 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:32.622533 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:32.622582 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:32.622668 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:22:32.623028 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.623593 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.624654 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.625065 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.625196 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:32.625249 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:32.625432 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 18:22:32.625509 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 18:22:32.625671 25591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:32.621761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.130:36155 (local address 127.23.72.129:39447)
0808 18:22:32.622219 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:32.622230 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:32.622370 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:32.622680 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:32.622683 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:32.622698 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:22:32.622907 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:32.622913 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.623702 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.623707 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:32.624533 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:32.624541 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.625192 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.625196 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:32.625296 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:32.625326 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 18:22:32.625417 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 18:22:32.625483 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"mutex_wait_us":128,"thread_start_us":126,"threads_started":1}
I20260808 18:22:32.625664 25581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:32.621894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:43551 (local address 127.23.72.131:41455)
0808 18:22:32.622050 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:32.622054 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:32.622289 (+ 235us) server_negotiation.cc:407] Connection header received
0808 18:22:32.622465 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:32.622468 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:32.622521 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:32.622609 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:22:32.622953 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.623510 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.624650 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.625071 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.625206 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:32.625260 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:32.625464 (+ 204us) server_negotiation.cc:299] Negotiation successful
0808 18:22:32.625537 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 18:22:32.625680 25592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:32.621760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.131:41455 (local address 127.23.72.129:43551)
0808 18:22:32.622177 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:32.622189 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:32.622329 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:32.622620 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:32.622623 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:32.622638 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:22:32.622836 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:32.622844 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.623627 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.623632 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:32.624524 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:32.624533 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.625197 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.625201 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:32.625327 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:32.625356 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 18:22:32.625445 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 18:22:32.625514 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"mutex_wait_us":26,"thread_start_us":159,"threads_started":1}
I20260808 18:22:32.626495 25356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1eb6806bfadb4de68152dc8d721ed66b" candidate_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a741b74692a642c697c799cdf11d6c27" is_pre_election: true
I20260808 18:22:32.626693 25507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1eb6806bfadb4de68152dc8d721ed66b" candidate_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a7890b8ca554c3192dcfefc94adcc48" is_pre_election: true
I20260808 18:22:32.626796 25356 raft_consensus.cc:2454] T 1eb6806bfadb4de68152dc8d721ed66b P a741b74692a642c697c799cdf11d6c27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5553dc2bdc4248aca8e7b3e808a7a95e in term 0.
I20260808 18:22:32.626984 25507 raft_consensus.cc:2454] T 1eb6806bfadb4de68152dc8d721ed66b P 2a7890b8ca554c3192dcfefc94adcc48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5553dc2bdc4248aca8e7b3e808a7a95e in term 0.
I20260808 18:22:32.627339 25137 leader_election.cc:302] T 1eb6806bfadb4de68152dc8d721ed66b P 5553dc2bdc4248aca8e7b3e808a7a95e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5553dc2bdc4248aca8e7b3e808a7a95e, a741b74692a642c697c799cdf11d6c27; no voters:
I20260808 18:22:32.627641 25590 raft_consensus.cc:2781] T 1eb6806bfadb4de68152dc8d721ed66b P 5553dc2bdc4248aca8e7b3e808a7a95e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:22:32.627729 25590 raft_consensus.cc:482] T 1eb6806bfadb4de68152dc8d721ed66b P 5553dc2bdc4248aca8e7b3e808a7a95e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:22:32.627774 25590 raft_consensus.cc:3037] T 1eb6806bfadb4de68152dc8d721ed66b P 5553dc2bdc4248aca8e7b3e808a7a95e [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:32.628819 25590 raft_consensus.cc:504] T 1eb6806bfadb4de68152dc8d721ed66b P 5553dc2bdc4248aca8e7b3e808a7a95e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a741b74692a642c697c799cdf11d6c27" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36155 } } peers { permanent_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41447 } } peers { permanent_uuid: "2a7890b8ca554c3192dcfefc94adcc48" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41455 } }
I20260808 18:22:32.629163 25590 leader_election.cc:288] T 1eb6806bfadb4de68152dc8d721ed66b P 5553dc2bdc4248aca8e7b3e808a7a95e [CANDIDATE]: Term 1 election: Requested vote from peers a741b74692a642c697c799cdf11d6c27 (127.23.72.130:36155), 2a7890b8ca554c3192dcfefc94adcc48 (127.23.72.131:41455)
I20260808 18:22:32.629467 25356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1eb6806bfadb4de68152dc8d721ed66b" candidate_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a741b74692a642c697c799cdf11d6c27"
I20260808 18:22:32.629549 25507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1eb6806bfadb4de68152dc8d721ed66b" candidate_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a7890b8ca554c3192dcfefc94adcc48"
I20260808 18:22:32.629598 25356 raft_consensus.cc:3037] T 1eb6806bfadb4de68152dc8d721ed66b P a741b74692a642c697c799cdf11d6c27 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:32.629669 25507 raft_consensus.cc:3037] T 1eb6806bfadb4de68152dc8d721ed66b P 2a7890b8ca554c3192dcfefc94adcc48 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:32.631502 25356 raft_consensus.cc:2454] T 1eb6806bfadb4de68152dc8d721ed66b P a741b74692a642c697c799cdf11d6c27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5553dc2bdc4248aca8e7b3e808a7a95e in term 1.
I20260808 18:22:32.631502 25507 raft_consensus.cc:2454] T 1eb6806bfadb4de68152dc8d721ed66b P 2a7890b8ca554c3192dcfefc94adcc48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5553dc2bdc4248aca8e7b3e808a7a95e in term 1.
I20260808 18:22:32.631810 25134 leader_election.cc:302] T 1eb6806bfadb4de68152dc8d721ed66b P 5553dc2bdc4248aca8e7b3e808a7a95e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a7890b8ca554c3192dcfefc94adcc48, 5553dc2bdc4248aca8e7b3e808a7a95e; no voters:
I20260808 18:22:32.631978 25590 raft_consensus.cc:2781] T 1eb6806bfadb4de68152dc8d721ed66b P 5553dc2bdc4248aca8e7b3e808a7a95e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:22:32.632220 25590 raft_consensus.cc:686] T 1eb6806bfadb4de68152dc8d721ed66b P 5553dc2bdc4248aca8e7b3e808a7a95e [term 1 LEADER]: Becoming Leader. State: Replica: 5553dc2bdc4248aca8e7b3e808a7a95e, State: Running, Role: LEADER
I20260808 18:22:32.632584 25590 consensus_queue.cc:237] T 1eb6806bfadb4de68152dc8d721ed66b P 5553dc2bdc4248aca8e7b3e808a7a95e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a741b74692a642c697c799cdf11d6c27" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36155 } } peers { permanent_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41447 } } peers { permanent_uuid: "2a7890b8ca554c3192dcfefc94adcc48" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41455 } }
I20260808 18:22:32.635798 25040 catalog_manager.cc:5495] T 1eb6806bfadb4de68152dc8d721ed66b P 5553dc2bdc4248aca8e7b3e808a7a95e reported cstate change: term changed from 0 to 1, leader changed from <none> to 5553dc2bdc4248aca8e7b3e808a7a95e (127.23.72.129). New cstate: current_term: 1 leader_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a741b74692a642c697c799cdf11d6c27" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41447 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2a7890b8ca554c3192dcfefc94adcc48" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41455 } health_report { overall_health: UNKNOWN } } }
I20260808 18:22:32.653666 25580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:32.649997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54030 (local address 127.23.72.129:41447)
0808 18:22:32.650148 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:32.650151 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:32.650165 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:22:32.650374 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:32.650376 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:32.650427 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:32.650506 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:22:32.650751 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.651244 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.652127 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.652347 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.652431 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:32.652710 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:22:32.653108 (+ 398us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:22:32.653346 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:32.653452 (+ 106us) server_negotiation.cc:299] Negotiation successful
0808 18:22:32.653500 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 18:22:32.659207 25507 raft_consensus.cc:1261] T 1eb6806bfadb4de68152dc8d721ed66b P 2a7890b8ca554c3192dcfefc94adcc48 [term 1 FOLLOWER]: Refusing update from remote peer 5553dc2bdc4248aca8e7b3e808a7a95e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:22:32.659219 25356 raft_consensus.cc:1261] T 1eb6806bfadb4de68152dc8d721ed66b P a741b74692a642c697c799cdf11d6c27 [term 1 FOLLOWER]: Refusing update from remote peer 5553dc2bdc4248aca8e7b3e808a7a95e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:22:32.659981 25593 consensus_queue.cc:1035] T 1eb6806bfadb4de68152dc8d721ed66b P 5553dc2bdc4248aca8e7b3e808a7a95e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a7890b8ca554c3192dcfefc94adcc48" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:22:32.660188 25590 consensus_queue.cc:1035] T 1eb6806bfadb4de68152dc8d721ed66b P 5553dc2bdc4248aca8e7b3e808a7a95e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a741b74692a642c697c799cdf11d6c27" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:22:32.667276 25596 mvcc.cc:205] Tried to move back new op lower bound from 7316329892485468160 to 7316329892385906688. Current Snapshot: MvccSnapshot[applied={T|T < 7316329892485468160}]
I20260808 18:22:32.679541 25042 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:22:32.679725 25042 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:22:32.682302 25042 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 01de4188925043e6810d12c91104af7b: Sending DeleteTablet for 3 replicas of tablet 1eb6806bfadb4de68152dc8d721ed66b
I20260808 18:22:32.683136 25333 tablet_service.cc:1514] Processing DeleteTablet for tablet 1eb6806bfadb4de68152dc8d721ed66b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:22:32 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:47212
I20260808 18:22:32.683156 25186 tablet_service.cc:1514] Processing DeleteTablet for tablet 1eb6806bfadb4de68152dc8d721ed66b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:22:32 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:54018
I20260808 18:22:32.683182 25484 tablet_service.cc:1514] Processing DeleteTablet for tablet 1eb6806bfadb4de68152dc8d721ed66b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:22:32 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:50062
I20260808 18:22:32.683804 25607 tablet_replica.cc:330] T 1eb6806bfadb4de68152dc8d721ed66b P a741b74692a642c697c799cdf11d6c27: stopping tablet replica
I20260808 18:22:32.683838 25606 tablet_replica.cc:330] T 1eb6806bfadb4de68152dc8d721ed66b P 2a7890b8ca554c3192dcfefc94adcc48: stopping tablet replica
I20260808 18:22:32.684127 25607 raft_consensus.cc:2229] T 1eb6806bfadb4de68152dc8d721ed66b P a741b74692a642c697c799cdf11d6c27 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:22:32.684275 25606 raft_consensus.cc:2229] T 1eb6806bfadb4de68152dc8d721ed66b P 2a7890b8ca554c3192dcfefc94adcc48 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:22:32.684448 25607 raft_consensus.cc:2258] T 1eb6806bfadb4de68152dc8d721ed66b P a741b74692a642c697c799cdf11d6c27 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:22:32.684695 25606 raft_consensus.cc:2258] T 1eb6806bfadb4de68152dc8d721ed66b P 2a7890b8ca554c3192dcfefc94adcc48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:22:32.684644 25040 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47418:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:22:32.685111 25040 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:22:32.686050 25605 tablet_replica.cc:330] T 1eb6806bfadb4de68152dc8d721ed66b P 5553dc2bdc4248aca8e7b3e808a7a95e: stopping tablet replica
I20260808 18:22:32.686084 25607 ts_tablet_manager.cc:1905] T 1eb6806bfadb4de68152dc8d721ed66b P a741b74692a642c697c799cdf11d6c27: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:22:32.686314 25605 raft_consensus.cc:2229] T 1eb6806bfadb4de68152dc8d721ed66b P 5553dc2bdc4248aca8e7b3e808a7a95e [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:22:32.686720 25605 raft_consensus.cc:2258] T 1eb6806bfadb4de68152dc8d721ed66b P 5553dc2bdc4248aca8e7b3e808a7a95e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:22:32.688093 25605 ts_tablet_manager.cc:1905] T 1eb6806bfadb4de68152dc8d721ed66b P 5553dc2bdc4248aca8e7b3e808a7a95e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:22:32.688925 25607 ts_tablet_manager.cc:1918] T 1eb6806bfadb4de68152dc8d721ed66b P a741b74692a642c697c799cdf11d6c27: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:22:32.689024 25607 log.cc:1196] T 1eb6806bfadb4de68152dc8d721ed66b P a741b74692a642c697c799cdf11d6c27: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-1/wal/wals/1eb6806bfadb4de68152dc8d721ed66b
I20260808 18:22:32.689324 25607 ts_tablet_manager.cc:1939] T 1eb6806bfadb4de68152dc8d721ed66b P a741b74692a642c697c799cdf11d6c27: Deleting consensus metadata
I20260808 18:22:32.690517 25029 catalog_manager.cc:4841] TS a741b74692a642c697c799cdf11d6c27 (127.23.72.130:36155): tablet 1eb6806bfadb4de68152dc8d721ed66b (table test-table [id=93b2fa4ca304476e9ba5556b1251db43]) successfully deleted
I20260808 18:22:32.690861 25605 ts_tablet_manager.cc:1918] T 1eb6806bfadb4de68152dc8d721ed66b P 5553dc2bdc4248aca8e7b3e808a7a95e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:22:32.690964 25605 log.cc:1196] T 1eb6806bfadb4de68152dc8d721ed66b P 5553dc2bdc4248aca8e7b3e808a7a95e: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-0/wal/wals/1eb6806bfadb4de68152dc8d721ed66b
I20260808 18:22:32.691315 25605 ts_tablet_manager.cc:1939] T 1eb6806bfadb4de68152dc8d721ed66b P 5553dc2bdc4248aca8e7b3e808a7a95e: Deleting consensus metadata
I20260808 18:22:32.692308 25186 tablet_service.cc:1467] Processing CreateTablet for tablet eb9fa6fac8be4df5baa0b4a67e6890ce (DEFAULT_TABLE table=test-table [id=b39f1b099fc241b6b3acd6f51c7b9157]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:22:32.692567 25186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb9fa6fac8be4df5baa0b4a67e6890ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:32.692531 25484 tablet_service.cc:1467] Processing CreateTablet for tablet eb9fa6fac8be4df5baa0b4a67e6890ce (DEFAULT_TABLE table=test-table [id=b39f1b099fc241b6b3acd6f51c7b9157]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:22:32.692791 25484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb9fa6fac8be4df5baa0b4a67e6890ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:32.693935 25606 ts_tablet_manager.cc:1905] T 1eb6806bfadb4de68152dc8d721ed66b P 2a7890b8ca554c3192dcfefc94adcc48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:22:32.694998 25333 tablet_service.cc:1467] Processing CreateTablet for tablet eb9fa6fac8be4df5baa0b4a67e6890ce (DEFAULT_TABLE table=test-table [id=b39f1b099fc241b6b3acd6f51c7b9157]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:22:32.695101 25584 tablet_bootstrap.cc:492] T eb9fa6fac8be4df5baa0b4a67e6890ce P 2a7890b8ca554c3192dcfefc94adcc48: Bootstrap starting.
I20260808 18:22:32.695242 25333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb9fa6fac8be4df5baa0b4a67e6890ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:32.695555 25583 tablet_bootstrap.cc:492] T eb9fa6fac8be4df5baa0b4a67e6890ce P 5553dc2bdc4248aca8e7b3e808a7a95e: Bootstrap starting.
I20260808 18:22:32.696069 25584 tablet_bootstrap.cc:654] T eb9fa6fac8be4df5baa0b4a67e6890ce P 2a7890b8ca554c3192dcfefc94adcc48: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:32.696544 25583 tablet_bootstrap.cc:654] T eb9fa6fac8be4df5baa0b4a67e6890ce P 5553dc2bdc4248aca8e7b3e808a7a95e: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:32.696547 25606 ts_tablet_manager.cc:1918] T 1eb6806bfadb4de68152dc8d721ed66b P 2a7890b8ca554c3192dcfefc94adcc48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:22:32.696625 25606 log.cc:1196] T 1eb6806bfadb4de68152dc8d721ed66b P 2a7890b8ca554c3192dcfefc94adcc48: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-2/wal/wals/1eb6806bfadb4de68152dc8d721ed66b
I20260808 18:22:32.696975 25606 ts_tablet_manager.cc:1939] T 1eb6806bfadb4de68152dc8d721ed66b P 2a7890b8ca554c3192dcfefc94adcc48: Deleting consensus metadata
I20260808 18:22:32.697320 25584 tablet_bootstrap.cc:492] T eb9fa6fac8be4df5baa0b4a67e6890ce P 2a7890b8ca554c3192dcfefc94adcc48: No bootstrap required, opened a new log
I20260808 18:22:32.697402 25584 ts_tablet_manager.cc:1397] T eb9fa6fac8be4df5baa0b4a67e6890ce P 2a7890b8ca554c3192dcfefc94adcc48: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 18:22:32.697512 25026 catalog_manager.cc:4841] TS 5553dc2bdc4248aca8e7b3e808a7a95e (127.23.72.129:41447): tablet 1eb6806bfadb4de68152dc8d721ed66b (table test-table [id=93b2fa4ca304476e9ba5556b1251db43]) successfully deleted
I20260808 18:22:32.697813 25584 raft_consensus.cc:348] T eb9fa6fac8be4df5baa0b4a67e6890ce P 2a7890b8ca554c3192dcfefc94adcc48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41447 } } peers { permanent_uuid: "a741b74692a642c697c799cdf11d6c27" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36155 } } peers { permanent_uuid: "2a7890b8ca554c3192dcfefc94adcc48" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41455 } }
I20260808 18:22:32.697917 25584 raft_consensus.cc:374] T eb9fa6fac8be4df5baa0b4a67e6890ce P 2a7890b8ca554c3192dcfefc94adcc48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:32.697947 25584 raft_consensus.cc:729] T eb9fa6fac8be4df5baa0b4a67e6890ce P 2a7890b8ca554c3192dcfefc94adcc48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a7890b8ca554c3192dcfefc94adcc48, State: Initialized, Role: FOLLOWER
I20260808 18:22:32.698060 25584 consensus_queue.cc:260] T eb9fa6fac8be4df5baa0b4a67e6890ce P 2a7890b8ca554c3192dcfefc94adcc48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41447 } } peers { permanent_uuid: "a741b74692a642c697c799cdf11d6c27" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36155 } } peers { permanent_uuid: "2a7890b8ca554c3192dcfefc94adcc48" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41455 } }
I20260808 18:22:32.698282 25584 ts_tablet_manager.cc:1428] T eb9fa6fac8be4df5baa0b4a67e6890ce P 2a7890b8ca554c3192dcfefc94adcc48: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:22:32.699116 25024 catalog_manager.cc:4841] TS 2a7890b8ca554c3192dcfefc94adcc48 (127.23.72.131:41455): tablet 1eb6806bfadb4de68152dc8d721ed66b (table test-table [id=93b2fa4ca304476e9ba5556b1251db43]) successfully deleted
I20260808 18:22:32.700389 25582 tablet_bootstrap.cc:492] T eb9fa6fac8be4df5baa0b4a67e6890ce P a741b74692a642c697c799cdf11d6c27: Bootstrap starting.
I20260808 18:22:32.700623 25583 tablet_bootstrap.cc:492] T eb9fa6fac8be4df5baa0b4a67e6890ce P 5553dc2bdc4248aca8e7b3e808a7a95e: No bootstrap required, opened a new log
I20260808 18:22:32.700711 25583 ts_tablet_manager.cc:1397] T eb9fa6fac8be4df5baa0b4a67e6890ce P 5553dc2bdc4248aca8e7b3e808a7a95e: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20260808 18:22:32.701191 25583 raft_consensus.cc:348] T eb9fa6fac8be4df5baa0b4a67e6890ce P 5553dc2bdc4248aca8e7b3e808a7a95e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41447 } } peers { permanent_uuid: "a741b74692a642c697c799cdf11d6c27" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36155 } } peers { permanent_uuid: "2a7890b8ca554c3192dcfefc94adcc48" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41455 } }
I20260808 18:22:32.701325 25583 raft_consensus.cc:374] T eb9fa6fac8be4df5baa0b4a67e6890ce P 5553dc2bdc4248aca8e7b3e808a7a95e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:32.701377 25583 raft_consensus.cc:729] T eb9fa6fac8be4df5baa0b4a67e6890ce P 5553dc2bdc4248aca8e7b3e808a7a95e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5553dc2bdc4248aca8e7b3e808a7a95e, State: Initialized, Role: FOLLOWER
I20260808 18:22:32.701408 25582 tablet_bootstrap.cc:654] T eb9fa6fac8be4df5baa0b4a67e6890ce P a741b74692a642c697c799cdf11d6c27: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:32.701514 25583 consensus_queue.cc:260] T eb9fa6fac8be4df5baa0b4a67e6890ce P 5553dc2bdc4248aca8e7b3e808a7a95e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41447 } } peers { permanent_uuid: "a741b74692a642c697c799cdf11d6c27" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36155 } } peers { permanent_uuid: "2a7890b8ca554c3192dcfefc94adcc48" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41455 } }
I20260808 18:22:32.702649 25582 tablet_bootstrap.cc:492] T eb9fa6fac8be4df5baa0b4a67e6890ce P a741b74692a642c697c799cdf11d6c27: No bootstrap required, opened a new log
I20260808 18:22:32.702726 25582 ts_tablet_manager.cc:1397] T eb9fa6fac8be4df5baa0b4a67e6890ce P a741b74692a642c697c799cdf11d6c27: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 18:22:32.702754 25583 ts_tablet_manager.cc:1428] T eb9fa6fac8be4df5baa0b4a67e6890ce P 5553dc2bdc4248aca8e7b3e808a7a95e: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260808 18:22:32.703109 25582 raft_consensus.cc:348] T eb9fa6fac8be4df5baa0b4a67e6890ce P a741b74692a642c697c799cdf11d6c27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41447 } } peers { permanent_uuid: "a741b74692a642c697c799cdf11d6c27" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36155 } } peers { permanent_uuid: "2a7890b8ca554c3192dcfefc94adcc48" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41455 } }
I20260808 18:22:32.703193 25582 raft_consensus.cc:374] T eb9fa6fac8be4df5baa0b4a67e6890ce P a741b74692a642c697c799cdf11d6c27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:32.703214 25582 raft_consensus.cc:729] T eb9fa6fac8be4df5baa0b4a67e6890ce P a741b74692a642c697c799cdf11d6c27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a741b74692a642c697c799cdf11d6c27, State: Initialized, Role: FOLLOWER
I20260808 18:22:32.703389 25582 consensus_queue.cc:260] T eb9fa6fac8be4df5baa0b4a67e6890ce P a741b74692a642c697c799cdf11d6c27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41447 } } peers { permanent_uuid: "a741b74692a642c697c799cdf11d6c27" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36155 } } peers { permanent_uuid: "2a7890b8ca554c3192dcfefc94adcc48" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41455 } }
I20260808 18:22:32.703677 25582 ts_tablet_manager.cc:1428] T eb9fa6fac8be4df5baa0b4a67e6890ce P a741b74692a642c697c799cdf11d6c27: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20260808 18:22:32.720563 25257 tablet.cc:2369] T eb9fa6fac8be4df5baa0b4a67e6890ce P 5553dc2bdc4248aca8e7b3e808a7a95e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:22:32.848981 25588 raft_consensus.cc:482] T eb9fa6fac8be4df5baa0b4a67e6890ce P a741b74692a642c697c799cdf11d6c27 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:22:32.849130 25588 raft_consensus.cc:504] T eb9fa6fac8be4df5baa0b4a67e6890ce P a741b74692a642c697c799cdf11d6c27 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41447 } } peers { permanent_uuid: "a741b74692a642c697c799cdf11d6c27" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36155 } } peers { permanent_uuid: "2a7890b8ca554c3192dcfefc94adcc48" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41455 } }
I20260808 18:22:32.850176 25588 leader_election.cc:288] T eb9fa6fac8be4df5baa0b4a67e6890ce P a741b74692a642c697c799cdf11d6c27 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5553dc2bdc4248aca8e7b3e808a7a95e (127.23.72.129:41447), 2a7890b8ca554c3192dcfefc94adcc48 (127.23.72.131:41455)
I20260808 18:22:32.855751 25580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:32.850628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:54493 (local address 127.23.72.129:41447)
0808 18:22:32.850770 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:32.850775 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:32.850872 (+ 97us) server_negotiation.cc:407] Connection header received
0808 18:22:32.851016 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:32.851019 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:32.851069 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:32.851157 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:22:32.851618 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.852412 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.854316 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.854947 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.855082 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:32.855135 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:32.855535 (+ 400us) server_negotiation.cc:299] Negotiation successful
0808 18:22:32.855626 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 18:22:32.855772 25581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:32.850549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:40355 (local address 127.23.72.131:41455)
0808 18:22:32.850773 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:32.850777 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:32.850937 (+ 160us) server_negotiation.cc:407] Connection header received
0808 18:22:32.851015 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:32.851018 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:32.851069 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:32.851157 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:22:32.851618 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.852412 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.854316 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.854985 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.855092 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:32.855135 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:32.855600 (+ 465us) server_negotiation.cc:299] Negotiation successful
0808 18:22:32.855670 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138}
I20260808 18:22:32.855810 25612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:32.850441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.131:41455 (local address 127.23.72.130:40355)
0808 18:22:32.850845 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:32.850857 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:32.850937 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:32.851195 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:32.851198 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:32.851242 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:22:32.851490 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:32.851496 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.852543 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.852547 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:32.854170 (+ 1623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:32.854181 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.855129 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.855134 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:32.855361 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:32.855411 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 18:22:32.855557 (+ 146us) client_negotiation.cc:240] Negotiation successful
0808 18:22:32.855608 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"mutex_wait_us":369,"spinlock_wait_cycles":28928,"thread_start_us":83,"threads_started":1}
I20260808 18:22:32.856211 25611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:32.850441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:41447 (local address 127.23.72.130:54493)
0808 18:22:32.850750 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:32.850777 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:32.850902 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:32.851195 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:32.851198 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:32.851243 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:22:32.851489 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:32.851496 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.852543 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.852547 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:32.854174 (+ 1627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:32.854181 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.855129 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.855134 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:32.855339 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:32.855410 (+ 71us) client_negotiation.cc:769] Sending connection context
0808 18:22:32.855518 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 18:22:32.855608 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"mutex_wait_us":284,"spinlock_wait_cycles":7680,"thread_start_us":91,"threads_started":1}
I20260808 18:22:32.856283 25507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb9fa6fac8be4df5baa0b4a67e6890ce" candidate_uuid: "a741b74692a642c697c799cdf11d6c27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a7890b8ca554c3192dcfefc94adcc48" is_pre_election: true
I20260808 18:22:32.856477 25507 raft_consensus.cc:2454] T eb9fa6fac8be4df5baa0b4a67e6890ce P 2a7890b8ca554c3192dcfefc94adcc48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a741b74692a642c697c799cdf11d6c27 in term 0.
I20260808 18:22:32.856771 25208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb9fa6fac8be4df5baa0b4a67e6890ce" candidate_uuid: "a741b74692a642c697c799cdf11d6c27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" is_pre_election: true
I20260808 18:22:32.856887 25282 leader_election.cc:302] T eb9fa6fac8be4df5baa0b4a67e6890ce P a741b74692a642c697c799cdf11d6c27 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a7890b8ca554c3192dcfefc94adcc48, a741b74692a642c697c799cdf11d6c27; no voters:
I20260808 18:22:32.857002 25208 raft_consensus.cc:2454] T eb9fa6fac8be4df5baa0b4a67e6890ce P 5553dc2bdc4248aca8e7b3e808a7a95e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a741b74692a642c697c799cdf11d6c27 in term 0.
I20260808 18:22:32.857173 25588 raft_consensus.cc:2781] T eb9fa6fac8be4df5baa0b4a67e6890ce P a741b74692a642c697c799cdf11d6c27 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:22:32.857234 25588 raft_consensus.cc:482] T eb9fa6fac8be4df5baa0b4a67e6890ce P a741b74692a642c697c799cdf11d6c27 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:22:32.857290 25588 raft_consensus.cc:3037] T eb9fa6fac8be4df5baa0b4a67e6890ce P a741b74692a642c697c799cdf11d6c27 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:32.858315 25588 raft_consensus.cc:504] T eb9fa6fac8be4df5baa0b4a67e6890ce P a741b74692a642c697c799cdf11d6c27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41447 } } peers { permanent_uuid: "a741b74692a642c697c799cdf11d6c27" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36155 } } peers { permanent_uuid: "2a7890b8ca554c3192dcfefc94adcc48" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41455 } }
I20260808 18:22:32.858664 25588 leader_election.cc:288] T eb9fa6fac8be4df5baa0b4a67e6890ce P a741b74692a642c697c799cdf11d6c27 [CANDIDATE]: Term 1 election: Requested vote from peers 5553dc2bdc4248aca8e7b3e808a7a95e (127.23.72.129:41447), 2a7890b8ca554c3192dcfefc94adcc48 (127.23.72.131:41455)
I20260808 18:22:32.859002 25208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb9fa6fac8be4df5baa0b4a67e6890ce" candidate_uuid: "a741b74692a642c697c799cdf11d6c27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e"
I20260808 18:22:32.859062 25507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb9fa6fac8be4df5baa0b4a67e6890ce" candidate_uuid: "a741b74692a642c697c799cdf11d6c27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a7890b8ca554c3192dcfefc94adcc48"
I20260808 18:22:32.859118 25208 raft_consensus.cc:3037] T eb9fa6fac8be4df5baa0b4a67e6890ce P 5553dc2bdc4248aca8e7b3e808a7a95e [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:32.859169 25507 raft_consensus.cc:3037] T eb9fa6fac8be4df5baa0b4a67e6890ce P 2a7890b8ca554c3192dcfefc94adcc48 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:32.860283 25208 raft_consensus.cc:2454] T eb9fa6fac8be4df5baa0b4a67e6890ce P 5553dc2bdc4248aca8e7b3e808a7a95e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a741b74692a642c697c799cdf11d6c27 in term 1.
I20260808 18:22:32.860283 25507 raft_consensus.cc:2454] T eb9fa6fac8be4df5baa0b4a67e6890ce P 2a7890b8ca554c3192dcfefc94adcc48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a741b74692a642c697c799cdf11d6c27 in term 1.
I20260808 18:22:32.860601 25283 leader_election.cc:302] T eb9fa6fac8be4df5baa0b4a67e6890ce P a741b74692a642c697c799cdf11d6c27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5553dc2bdc4248aca8e7b3e808a7a95e, a741b74692a642c697c799cdf11d6c27; no voters:
I20260808 18:22:32.860782 25588 raft_consensus.cc:2781] T eb9fa6fac8be4df5baa0b4a67e6890ce P a741b74692a642c697c799cdf11d6c27 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:22:32.861004 25588 raft_consensus.cc:686] T eb9fa6fac8be4df5baa0b4a67e6890ce P a741b74692a642c697c799cdf11d6c27 [term 1 LEADER]: Becoming Leader. State: Replica: a741b74692a642c697c799cdf11d6c27, State: Running, Role: LEADER
I20260808 18:22:32.861253 25588 consensus_queue.cc:237] T eb9fa6fac8be4df5baa0b4a67e6890ce P a741b74692a642c697c799cdf11d6c27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41447 } } peers { permanent_uuid: "a741b74692a642c697c799cdf11d6c27" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36155 } } peers { permanent_uuid: "2a7890b8ca554c3192dcfefc94adcc48" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41455 } }
I20260808 18:22:32.863729 25042 catalog_manager.cc:5495] T eb9fa6fac8be4df5baa0b4a67e6890ce P a741b74692a642c697c799cdf11d6c27 reported cstate change: term changed from 0 to 1, leader changed from <none> to a741b74692a642c697c799cdf11d6c27 (127.23.72.130). New cstate: current_term: 1 leader_uuid: "a741b74692a642c697c799cdf11d6c27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a741b74692a642c697c799cdf11d6c27" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2a7890b8ca554c3192dcfefc94adcc48" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41455 } health_report { overall_health: UNKNOWN } } }
Aug 08 18:22:32 dist-test-slave-5k54 krb5kdc[24995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213351, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:22:32.917342 25620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:32.909121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:40501 (local address 127.0.0.1:47428)
0808 18:22:32.909413 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:32.909426 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:32.909564 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:32.909816 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:32.909819 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:32.910242 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:32.910338 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:32.910345 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.911093 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.911099 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:32.911739 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:32.911748 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.912182 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.912185 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:32.912261 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:32.912869 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:32.912881 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:32.914292 (+ 1411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:32.915978 (+ 1686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:32.915986 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:32.915991 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:32.916217 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:32.916452 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:32.916454 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:32.916455 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:32.916533 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:32.916871 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:32.916876 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:32.916982 (+ 106us) client_negotiation.cc:769] Sending connection context
0808 18:22:32.917070 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 18:22:32.917176 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":82,"threads_started":1}
I20260808 18:22:32.917352 25094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:32.909200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47428 (local address 127.23.72.190:40501)
0808 18:22:32.909335 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:32.909340 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:32.909510 (+ 170us) server_negotiation.cc:407] Connection header received
0808 18:22:32.909669 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:32.909672 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:32.909717 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:32.909800 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:32.910467 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.910987 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.911856 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:32.912074 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:32.912180 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:32.914433 (+ 2253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:32.914436 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:32.914438 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:32.914461 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:32.915883 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:32.916298 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:32.916301 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:32.916302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:32.916375 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:32.916611 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:32.916613 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:32.916614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:32.916784 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:32.916890 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:32.917128 (+ 238us) server_negotiation.cc:299] Negotiation successful
0808 18:22:32.917222 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 18:22:32.921264 25042 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:47428:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 18:22:32.921991 25042 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:22:32.926898 25186 tablet_service.cc:1467] Processing CreateTablet for tablet 88630d35db9c41aeacccba8a0473c9f8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9a2ea546bd9849d1a558aa79eccc4f1e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 18:22:32.926981 25333 tablet_service.cc:1467] Processing CreateTablet for tablet 88630d35db9c41aeacccba8a0473c9f8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9a2ea546bd9849d1a558aa79eccc4f1e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 18:22:32.927043 25484 tablet_service.cc:1467] Processing CreateTablet for tablet 88630d35db9c41aeacccba8a0473c9f8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9a2ea546bd9849d1a558aa79eccc4f1e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 18:22:32.927311 25333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88630d35db9c41aeacccba8a0473c9f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:32.927320 25186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88630d35db9c41aeacccba8a0473c9f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:32.927475 25484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88630d35db9c41aeacccba8a0473c9f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:32.929904 25584 tablet_bootstrap.cc:492] T 88630d35db9c41aeacccba8a0473c9f8 P 2a7890b8ca554c3192dcfefc94adcc48: Bootstrap starting.
I20260808 18:22:32.930186 25583 tablet_bootstrap.cc:492] T 88630d35db9c41aeacccba8a0473c9f8 P 5553dc2bdc4248aca8e7b3e808a7a95e: Bootstrap starting.
I20260808 18:22:32.930357 25582 tablet_bootstrap.cc:492] T 88630d35db9c41aeacccba8a0473c9f8 P a741b74692a642c697c799cdf11d6c27: Bootstrap starting.
I20260808 18:22:32.930998 25583 tablet_bootstrap.cc:654] T 88630d35db9c41aeacccba8a0473c9f8 P 5553dc2bdc4248aca8e7b3e808a7a95e: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:32.930998 25584 tablet_bootstrap.cc:654] T 88630d35db9c41aeacccba8a0473c9f8 P 2a7890b8ca554c3192dcfefc94adcc48: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:32.931288 25582 tablet_bootstrap.cc:654] T 88630d35db9c41aeacccba8a0473c9f8 P a741b74692a642c697c799cdf11d6c27: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:32.932420 25584 tablet_bootstrap.cc:492] T 88630d35db9c41aeacccba8a0473c9f8 P 2a7890b8ca554c3192dcfefc94adcc48: No bootstrap required, opened a new log
I20260808 18:22:32.932515 25584 ts_tablet_manager.cc:1397] T 88630d35db9c41aeacccba8a0473c9f8 P 2a7890b8ca554c3192dcfefc94adcc48: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 18:22:32.932827 25582 tablet_bootstrap.cc:492] T 88630d35db9c41aeacccba8a0473c9f8 P a741b74692a642c697c799cdf11d6c27: No bootstrap required, opened a new log
I20260808 18:22:32.932934 25582 ts_tablet_manager.cc:1397] T 88630d35db9c41aeacccba8a0473c9f8 P a741b74692a642c697c799cdf11d6c27: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 18:22:32.933010 25584 raft_consensus.cc:348] T 88630d35db9c41aeacccba8a0473c9f8 P 2a7890b8ca554c3192dcfefc94adcc48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41447 } } peers { permanent_uuid: "a741b74692a642c697c799cdf11d6c27" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36155 } } peers { permanent_uuid: "2a7890b8ca554c3192dcfefc94adcc48" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41455 } }
I20260808 18:22:32.933147 25584 raft_consensus.cc:374] T 88630d35db9c41aeacccba8a0473c9f8 P 2a7890b8ca554c3192dcfefc94adcc48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:32.933125 25583 tablet_bootstrap.cc:492] T 88630d35db9c41aeacccba8a0473c9f8 P 5553dc2bdc4248aca8e7b3e808a7a95e: No bootstrap required, opened a new log
I20260808 18:22:32.933182 25584 raft_consensus.cc:729] T 88630d35db9c41aeacccba8a0473c9f8 P 2a7890b8ca554c3192dcfefc94adcc48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a7890b8ca554c3192dcfefc94adcc48, State: Initialized, Role: FOLLOWER
I20260808 18:22:32.933225 25583 ts_tablet_manager.cc:1397] T 88630d35db9c41aeacccba8a0473c9f8 P 5553dc2bdc4248aca8e7b3e808a7a95e: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260808 18:22:32.933317 25584 consensus_queue.cc:260] T 88630d35db9c41aeacccba8a0473c9f8 P 2a7890b8ca554c3192dcfefc94adcc48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41447 } } peers { permanent_uuid: "a741b74692a642c697c799cdf11d6c27" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36155 } } peers { permanent_uuid: "2a7890b8ca554c3192dcfefc94adcc48" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41455 } }
I20260808 18:22:32.933396 25582 raft_consensus.cc:348] T 88630d35db9c41aeacccba8a0473c9f8 P a741b74692a642c697c799cdf11d6c27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41447 } } peers { permanent_uuid: "a741b74692a642c697c799cdf11d6c27" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36155 } } peers { permanent_uuid: "2a7890b8ca554c3192dcfefc94adcc48" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41455 } }
I20260808 18:22:32.933490 25582 raft_consensus.cc:374] T 88630d35db9c41aeacccba8a0473c9f8 P a741b74692a642c697c799cdf11d6c27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:32.933528 25582 raft_consensus.cc:729] T 88630d35db9c41aeacccba8a0473c9f8 P a741b74692a642c697c799cdf11d6c27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a741b74692a642c697c799cdf11d6c27, State: Initialized, Role: FOLLOWER
I20260808 18:22:32.933672 25584 ts_tablet_manager.cc:1428] T 88630d35db9c41aeacccba8a0473c9f8 P 2a7890b8ca554c3192dcfefc94adcc48: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:22:32.933642 25582 consensus_queue.cc:260] T 88630d35db9c41aeacccba8a0473c9f8 P a741b74692a642c697c799cdf11d6c27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41447 } } peers { permanent_uuid: "a741b74692a642c697c799cdf11d6c27" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36155 } } peers { permanent_uuid: "2a7890b8ca554c3192dcfefc94adcc48" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41455 } }
I20260808 18:22:32.933640 25583 raft_consensus.cc:348] T 88630d35db9c41aeacccba8a0473c9f8 P 5553dc2bdc4248aca8e7b3e808a7a95e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41447 } } peers { permanent_uuid: "a741b74692a642c697c799cdf11d6c27" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36155 } } peers { permanent_uuid: "2a7890b8ca554c3192dcfefc94adcc48" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41455 } }
I20260808 18:22:32.933784 25583 raft_consensus.cc:374] T 88630d35db9c41aeacccba8a0473c9f8 P 5553dc2bdc4248aca8e7b3e808a7a95e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:32.933838 25583 raft_consensus.cc:729] T 88630d35db9c41aeacccba8a0473c9f8 P 5553dc2bdc4248aca8e7b3e808a7a95e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5553dc2bdc4248aca8e7b3e808a7a95e, State: Initialized, Role: FOLLOWER
I20260808 18:22:32.933911 25582 ts_tablet_manager.cc:1428] T 88630d35db9c41aeacccba8a0473c9f8 P a741b74692a642c697c799cdf11d6c27: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:22:32.933916 25589 tablet_replica.cc:439] T 88630d35db9c41aeacccba8a0473c9f8 P 2a7890b8ca554c3192dcfefc94adcc48: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41447 } } peers { permanent_uuid: "a741b74692a642c697c799cdf11d6c27" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36155 } } peers { permanent_uuid: "2a7890b8ca554c3192dcfefc94adcc48" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41455 } } }
I20260808 18:22:32.933979 25583 consensus_queue.cc:260] T 88630d35db9c41aeacccba8a0473c9f8 P 5553dc2bdc4248aca8e7b3e808a7a95e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41447 } } peers { permanent_uuid: "a741b74692a642c697c799cdf11d6c27" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36155 } } peers { permanent_uuid: "2a7890b8ca554c3192dcfefc94adcc48" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41455 } }
I20260808 18:22:32.934072 25589 tablet_replica.cc:442] T 88630d35db9c41aeacccba8a0473c9f8 P 2a7890b8ca554c3192dcfefc94adcc48: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:22:32.934098 25588 tablet_replica.cc:439] T 88630d35db9c41aeacccba8a0473c9f8 P a741b74692a642c697c799cdf11d6c27: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41447 } } peers { permanent_uuid: "a741b74692a642c697c799cdf11d6c27" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36155 } } peers { permanent_uuid: "2a7890b8ca554c3192dcfefc94adcc48" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41455 } } }
I20260808 18:22:32.934232 25588 tablet_replica.cc:442] T 88630d35db9c41aeacccba8a0473c9f8 P a741b74692a642c697c799cdf11d6c27: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:22:32.934290 25583 ts_tablet_manager.cc:1428] T 88630d35db9c41aeacccba8a0473c9f8 P 5553dc2bdc4248aca8e7b3e808a7a95e: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260808 18:22:32.934469 25593 tablet_replica.cc:439] T 88630d35db9c41aeacccba8a0473c9f8 P 5553dc2bdc4248aca8e7b3e808a7a95e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41447 } } peers { permanent_uuid: "a741b74692a642c697c799cdf11d6c27" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36155 } } peers { permanent_uuid: "2a7890b8ca554c3192dcfefc94adcc48" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41455 } } }
I20260808 18:22:32.934594 25593 tablet_replica.cc:442] T 88630d35db9c41aeacccba8a0473c9f8 P 5553dc2bdc4248aca8e7b3e808a7a95e: This TxnStatusTablet replica's current role is: FOLLOWER
W20260808 18:22:32.936105 25557 tablet.cc:2369] T 88630d35db9c41aeacccba8a0473c9f8 P 2a7890b8ca554c3192dcfefc94adcc48: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 18:22:32.953303 25406 tablet.cc:2369] T 88630d35db9c41aeacccba8a0473c9f8 P a741b74692a642c697c799cdf11d6c27: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:22:32.995487 25593 raft_consensus.cc:482] T 88630d35db9c41aeacccba8a0473c9f8 P 5553dc2bdc4248aca8e7b3e808a7a95e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:22:32.995621 25593 raft_consensus.cc:504] T 88630d35db9c41aeacccba8a0473c9f8 P 5553dc2bdc4248aca8e7b3e808a7a95e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41447 } } peers { permanent_uuid: "a741b74692a642c697c799cdf11d6c27" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36155 } } peers { permanent_uuid: "2a7890b8ca554c3192dcfefc94adcc48" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41455 } }
I20260808 18:22:32.996086 25593 leader_election.cc:288] T 88630d35db9c41aeacccba8a0473c9f8 P 5553dc2bdc4248aca8e7b3e808a7a95e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a741b74692a642c697c799cdf11d6c27 (127.23.72.130:36155), 2a7890b8ca554c3192dcfefc94adcc48 (127.23.72.131:41455)
I20260808 18:22:32.996501 25355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88630d35db9c41aeacccba8a0473c9f8" candidate_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a741b74692a642c697c799cdf11d6c27" is_pre_election: true
I20260808 18:22:32.996584 25507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88630d35db9c41aeacccba8a0473c9f8" candidate_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a7890b8ca554c3192dcfefc94adcc48" is_pre_election: true
I20260808 18:22:32.996672 25355 raft_consensus.cc:2454] T 88630d35db9c41aeacccba8a0473c9f8 P a741b74692a642c697c799cdf11d6c27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5553dc2bdc4248aca8e7b3e808a7a95e in term 0.
I20260808 18:22:32.996780 25507 raft_consensus.cc:2454] T 88630d35db9c41aeacccba8a0473c9f8 P 2a7890b8ca554c3192dcfefc94adcc48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5553dc2bdc4248aca8e7b3e808a7a95e in term 0.
I20260808 18:22:32.997020 25137 leader_election.cc:302] T 88630d35db9c41aeacccba8a0473c9f8 P 5553dc2bdc4248aca8e7b3e808a7a95e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5553dc2bdc4248aca8e7b3e808a7a95e, a741b74692a642c697c799cdf11d6c27; no voters:
I20260808 18:22:32.997195 25593 raft_consensus.cc:2781] T 88630d35db9c41aeacccba8a0473c9f8 P 5553dc2bdc4248aca8e7b3e808a7a95e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:22:32.997259 25593 raft_consensus.cc:482] T 88630d35db9c41aeacccba8a0473c9f8 P 5553dc2bdc4248aca8e7b3e808a7a95e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:22:32.997303 25593 raft_consensus.cc:3037] T 88630d35db9c41aeacccba8a0473c9f8 P 5553dc2bdc4248aca8e7b3e808a7a95e [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:32.998360 25593 raft_consensus.cc:504] T 88630d35db9c41aeacccba8a0473c9f8 P 5553dc2bdc4248aca8e7b3e808a7a95e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41447 } } peers { permanent_uuid: "a741b74692a642c697c799cdf11d6c27" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36155 } } peers { permanent_uuid: "2a7890b8ca554c3192dcfefc94adcc48" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41455 } }
I20260808 18:22:32.998731 25593 leader_election.cc:288] T 88630d35db9c41aeacccba8a0473c9f8 P 5553dc2bdc4248aca8e7b3e808a7a95e [CANDIDATE]: Term 1 election: Requested vote from peers a741b74692a642c697c799cdf11d6c27 (127.23.72.130:36155), 2a7890b8ca554c3192dcfefc94adcc48 (127.23.72.131:41455)
I20260808 18:22:32.999040 25355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88630d35db9c41aeacccba8a0473c9f8" candidate_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a741b74692a642c697c799cdf11d6c27"
I20260808 18:22:32.999150 25507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88630d35db9c41aeacccba8a0473c9f8" candidate_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a7890b8ca554c3192dcfefc94adcc48"
I20260808 18:22:32.999183 25355 raft_consensus.cc:3037] T 88630d35db9c41aeacccba8a0473c9f8 P a741b74692a642c697c799cdf11d6c27 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:32.999264 25507 raft_consensus.cc:3037] T 88630d35db9c41aeacccba8a0473c9f8 P 2a7890b8ca554c3192dcfefc94adcc48 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:33.000468 25507 raft_consensus.cc:2454] T 88630d35db9c41aeacccba8a0473c9f8 P 2a7890b8ca554c3192dcfefc94adcc48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5553dc2bdc4248aca8e7b3e808a7a95e in term 1.
I20260808 18:22:33.000535 25355 raft_consensus.cc:2454] T 88630d35db9c41aeacccba8a0473c9f8 P a741b74692a642c697c799cdf11d6c27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5553dc2bdc4248aca8e7b3e808a7a95e in term 1.
I20260808 18:22:33.000766 25134 leader_election.cc:302] T 88630d35db9c41aeacccba8a0473c9f8 P 5553dc2bdc4248aca8e7b3e808a7a95e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a7890b8ca554c3192dcfefc94adcc48, 5553dc2bdc4248aca8e7b3e808a7a95e; no voters:
I20260808 18:22:33.000953 25593 raft_consensus.cc:2781] T 88630d35db9c41aeacccba8a0473c9f8 P 5553dc2bdc4248aca8e7b3e808a7a95e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:22:33.001046 25593 raft_consensus.cc:686] T 88630d35db9c41aeacccba8a0473c9f8 P 5553dc2bdc4248aca8e7b3e808a7a95e [term 1 LEADER]: Becoming Leader. State: Replica: 5553dc2bdc4248aca8e7b3e808a7a95e, State: Running, Role: LEADER
I20260808 18:22:33.001225 25593 consensus_queue.cc:237] T 88630d35db9c41aeacccba8a0473c9f8 P 5553dc2bdc4248aca8e7b3e808a7a95e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41447 } } peers { permanent_uuid: "a741b74692a642c697c799cdf11d6c27" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36155 } } peers { permanent_uuid: "2a7890b8ca554c3192dcfefc94adcc48" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41455 } }
I20260808 18:22:33.002005 25590 tablet_replica.cc:439] T 88630d35db9c41aeacccba8a0473c9f8 P 5553dc2bdc4248aca8e7b3e808a7a95e: TxnStatusTablet state changed. Reason: New leader 5553dc2bdc4248aca8e7b3e808a7a95e. Latest consensus state: current_term: 1 leader_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41447 } } peers { permanent_uuid: "a741b74692a642c697c799cdf11d6c27" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36155 } } peers { permanent_uuid: "2a7890b8ca554c3192dcfefc94adcc48" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41455 } } }
I20260808 18:22:33.002120 25590 tablet_replica.cc:442] T 88630d35db9c41aeacccba8a0473c9f8 P 5553dc2bdc4248aca8e7b3e808a7a95e: This TxnStatusTablet replica's current role is: LEADER
I20260808 18:22:33.002444 25626 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 18:22:33.002638 25626 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 18:22:33.003160 25040 catalog_manager.cc:5495] T 88630d35db9c41aeacccba8a0473c9f8 P 5553dc2bdc4248aca8e7b3e808a7a95e reported cstate change: term changed from 0 to 1, leader changed from <none> to 5553dc2bdc4248aca8e7b3e808a7a95e (127.23.72.129). New cstate: current_term: 1 leader_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41447 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a741b74692a642c697c799cdf11d6c27" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a7890b8ca554c3192dcfefc94adcc48" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41455 } health_report { overall_health: UNKNOWN } } }
I20260808 18:22:33.019836 25580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:33.016104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54046 (local address 127.23.72.129:41447)
0808 18:22:33.016253 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:33.016257 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:33.016363 (+ 106us) server_negotiation.cc:407] Connection header received
0808 18:22:33.016539 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:33.016541 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:33.016584 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:33.016647 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:22:33.016963 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:33.017471 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:33.018678 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:33.018924 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:33.019006 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:33.019330 (+ 324us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:22:33.019403 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:22:33.019519 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:33.019637 (+ 118us) server_negotiation.cc:299] Negotiation successful
0808 18:22:33.019687 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 18:22:33.019851 25620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:33.015997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:41447 (local address 127.0.0.1:54046)
0808 18:22:33.016278 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:33.016294 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:33.016421 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:33.016694 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:33.016697 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:33.016705 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 18:22:33.016820 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:33.016826 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:33.017580 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:33.017583 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:33.018556 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:33.018567 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:33.019033 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:33.019037 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:33.019129 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:33.019186 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 18:22:33.019538 (+ 352us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 18:22:33.019554 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 18:22:33.019621 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 18:22:33.019710 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167}
I20260808 18:22:33.022294 25355 raft_consensus.cc:1261] T 88630d35db9c41aeacccba8a0473c9f8 P a741b74692a642c697c799cdf11d6c27 [term 1 FOLLOWER]: Refusing update from remote peer 5553dc2bdc4248aca8e7b3e808a7a95e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:22:33.022295 25507 raft_consensus.cc:1261] T 88630d35db9c41aeacccba8a0473c9f8 P 2a7890b8ca554c3192dcfefc94adcc48 [term 1 FOLLOWER]: Refusing update from remote peer 5553dc2bdc4248aca8e7b3e808a7a95e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:22:33.022737 25590 consensus_queue.cc:1035] T 88630d35db9c41aeacccba8a0473c9f8 P 5553dc2bdc4248aca8e7b3e808a7a95e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a7890b8ca554c3192dcfefc94adcc48" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:22:33.022918 25593 consensus_queue.cc:1035] T 88630d35db9c41aeacccba8a0473c9f8 P 5553dc2bdc4248aca8e7b3e808a7a95e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a741b74692a642c697c799cdf11d6c27" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:22:33.024302 25588 tablet_replica.cc:439] T 88630d35db9c41aeacccba8a0473c9f8 P a741b74692a642c697c799cdf11d6c27: TxnStatusTablet state changed. Reason: New leader 5553dc2bdc4248aca8e7b3e808a7a95e. Latest consensus state: current_term: 1 leader_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41447 } } peers { permanent_uuid: "a741b74692a642c697c799cdf11d6c27" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36155 } } peers { permanent_uuid: "2a7890b8ca554c3192dcfefc94adcc48" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41455 } } }
I20260808 18:22:33.024420 25588 tablet_replica.cc:442] T 88630d35db9c41aeacccba8a0473c9f8 P a741b74692a642c697c799cdf11d6c27: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:22:33.024507 25589 tablet_replica.cc:439] T 88630d35db9c41aeacccba8a0473c9f8 P 2a7890b8ca554c3192dcfefc94adcc48: TxnStatusTablet state changed. Reason: New leader 5553dc2bdc4248aca8e7b3e808a7a95e. Latest consensus state: current_term: 1 leader_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41447 } } peers { permanent_uuid: "a741b74692a642c697c799cdf11d6c27" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36155 } } peers { permanent_uuid: "2a7890b8ca554c3192dcfefc94adcc48" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41455 } } }
I20260808 18:22:33.024618 25589 tablet_replica.cc:442] T 88630d35db9c41aeacccba8a0473c9f8 P 2a7890b8ca554c3192dcfefc94adcc48: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:22:33.025949 25590 tablet_replica.cc:439] T 88630d35db9c41aeacccba8a0473c9f8 P 5553dc2bdc4248aca8e7b3e808a7a95e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41447 } } peers { permanent_uuid: "a741b74692a642c697c799cdf11d6c27" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36155 } } peers { permanent_uuid: "2a7890b8ca554c3192dcfefc94adcc48" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41455 } } }
I20260808 18:22:33.026055 25590 tablet_replica.cc:442] T 88630d35db9c41aeacccba8a0473c9f8 P 5553dc2bdc4248aca8e7b3e808a7a95e: This TxnStatusTablet replica's current role is: LEADER
I20260808 18:22:33.026459 25597 mvcc.cc:205] Tried to move back new op lower bound from 7316329893975326720 to 7316329893894742016. Current Snapshot: MvccSnapshot[applied={T|T < 7316329893975326720}]
I20260808 18:22:33.027119 25589 tablet_replica.cc:439] T 88630d35db9c41aeacccba8a0473c9f8 P 2a7890b8ca554c3192dcfefc94adcc48: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41447 } } peers { permanent_uuid: "a741b74692a642c697c799cdf11d6c27" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36155 } } peers { permanent_uuid: "2a7890b8ca554c3192dcfefc94adcc48" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41455 } } }
I20260808 18:22:33.027215 25589 tablet_replica.cc:442] T 88630d35db9c41aeacccba8a0473c9f8 P 2a7890b8ca554c3192dcfefc94adcc48: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:22:33.029700 25593 tablet_replica.cc:439] T 88630d35db9c41aeacccba8a0473c9f8 P 5553dc2bdc4248aca8e7b3e808a7a95e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41447 } } peers { permanent_uuid: "a741b74692a642c697c799cdf11d6c27" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36155 } } peers { permanent_uuid: "2a7890b8ca554c3192dcfefc94adcc48" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41455 } } }
I20260808 18:22:33.029812 25593 tablet_replica.cc:442] T 88630d35db9c41aeacccba8a0473c9f8 P 5553dc2bdc4248aca8e7b3e808a7a95e: This TxnStatusTablet replica's current role is: LEADER
I20260808 18:22:33.030930 25588 tablet_replica.cc:439] T 88630d35db9c41aeacccba8a0473c9f8 P a741b74692a642c697c799cdf11d6c27: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41447 } } peers { permanent_uuid: "a741b74692a642c697c799cdf11d6c27" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36155 } } peers { permanent_uuid: "2a7890b8ca554c3192dcfefc94adcc48" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41455 } } }
I20260808 18:22:33.031044 25588 tablet_replica.cc:442] T 88630d35db9c41aeacccba8a0473c9f8 P a741b74692a642c697c799cdf11d6c27: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:22:33.032595 25598 mvcc.cc:205] Tried to move back new op lower bound from 7316329893975326720 to 7316329893894742016. Current Snapshot: MvccSnapshot[applied={T|T < 7316329893975326720 or (T in {7316329893975326720})}]
I20260808 18:22:33.036154 25577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:33.032088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47228 (local address 127.23.72.130:36155)
0808 18:22:33.032198 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:33.032202 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:33.032215 (+ 13us) server_negotiation.cc:407] Connection header received
0808 18:22:33.032287 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:33.032289 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:33.032335 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:33.032449 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:22:33.032737 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:33.033455 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:33.034642 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:33.034884 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:33.034968 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:33.035253 (+ 285us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:22:33.035763 (+ 510us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:22:33.035864 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:33.035962 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 18:22:33.036010 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 18:22:33.048043 25580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:33.044401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:54005 (local address 127.23.72.129:41447)
0808 18:22:33.044548 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:33.044554 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:33.044712 (+ 158us) server_negotiation.cc:407] Connection header received
0808 18:22:33.044872 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:33.044874 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:33.044919 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:33.044975 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:22:33.045265 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:33.045750 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:33.046981 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:33.047198 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:33.047282 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:33.047650 (+ 368us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:22:33.047717 (+ 67us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:22:33.047771 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:33.047882 (+ 111us) server_negotiation.cc:299] Negotiation successful
0808 18:22:33.047930 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 18:22:33.048126 25634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:33.044303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:41447 (local address 127.23.72.130:54005)
0808 18:22:33.044641 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:33.044657 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:33.044776 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:33.045017 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:33.045020 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:33.045028 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 18:22:33.045176 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:33.045182 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:33.045858 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:33.045861 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:33.046849 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:33.046860 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:33.047340 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:33.047344 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:33.047467 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:33.047520 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 18:22:33.047803 (+ 283us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 18:22:33.047809 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 18:22:33.047874 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 18:22:33.047955 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":118,"threads_started":1}
I20260808 18:22:33.054980 25507 raft_consensus.cc:1261] T eb9fa6fac8be4df5baa0b4a67e6890ce P 2a7890b8ca554c3192dcfefc94adcc48 [term 1 FOLLOWER]: Refusing update from remote peer a741b74692a642c697c799cdf11d6c27: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:22:33.055122 25208 raft_consensus.cc:1261] T eb9fa6fac8be4df5baa0b4a67e6890ce P 5553dc2bdc4248aca8e7b3e808a7a95e [term 1 FOLLOWER]: Refusing update from remote peer a741b74692a642c697c799cdf11d6c27: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:22:33.055531 25613 consensus_queue.cc:1035] T eb9fa6fac8be4df5baa0b4a67e6890ce P a741b74692a642c697c799cdf11d6c27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a7890b8ca554c3192dcfefc94adcc48" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:22:33.055742 25588 consensus_queue.cc:1035] T eb9fa6fac8be4df5baa0b4a67e6890ce P a741b74692a642c697c799cdf11d6c27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5553dc2bdc4248aca8e7b3e808a7a95e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:22:33.080106 25642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:33.076334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.130:36155 (local address 127.23.72.129:35645)
0808 18:22:33.076748 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:33.076761 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:33.076851 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:33.077102 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:33.077104 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:33.077112 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 18:22:33.077230 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:33.077241 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:33.078019 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:33.078023 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:33.078795 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:33.078804 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:33.079335 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:33.079338 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:33.079457 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:33.079501 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 18:22:33.079811 (+ 310us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 18:22:33.079820 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 18:22:33.079896 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 18:22:33.079976 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":110,"threads_started":1}
I20260808 18:22:33.080090 25577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:33.076418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:35645 (local address 127.23.72.130:36155)
0808 18:22:33.076569 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:33.076574 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:33.076839 (+ 265us) server_negotiation.cc:407] Connection header received
0808 18:22:33.076924 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:33.076927 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:33.076979 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:33.077077 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:22:33.077355 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:33.077886 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:33.078969 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:33.079223 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:33.079332 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:33.079658 (+ 326us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:22:33.079728 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:22:33.079806 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:33.079904 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 18:22:33.079955 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 18:22:33.108270 25040 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:22:33.108471 25040 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:22:33.110765 25040 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 01de4188925043e6810d12c91104af7b: Sending DeleteTablet for 3 replicas of tablet eb9fa6fac8be4df5baa0b4a67e6890ce
I20260808 18:22:33.111364 25186 tablet_service.cc:1514] Processing DeleteTablet for tablet eb9fa6fac8be4df5baa0b4a67e6890ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:22:33 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:54018
I20260808 18:22:33.111608 25605 tablet_replica.cc:330] T eb9fa6fac8be4df5baa0b4a67e6890ce P 5553dc2bdc4248aca8e7b3e808a7a95e: stopping tablet replica
I20260808 18:22:33.111585 25484 tablet_service.cc:1514] Processing DeleteTablet for tablet eb9fa6fac8be4df5baa0b4a67e6890ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:22:33 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:50062
I20260808 18:22:33.111752 25605 raft_consensus.cc:2229] T eb9fa6fac8be4df5baa0b4a67e6890ce P 5553dc2bdc4248aca8e7b3e808a7a95e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:22:33.111907 25605 raft_consensus.cc:2258] T eb9fa6fac8be4df5baa0b4a67e6890ce P 5553dc2bdc4248aca8e7b3e808a7a95e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:22:33.112115 25606 tablet_replica.cc:330] T eb9fa6fac8be4df5baa0b4a67e6890ce P 2a7890b8ca554c3192dcfefc94adcc48: stopping tablet replica
I20260808 18:22:33.112210 25333 tablet_service.cc:1514] Processing DeleteTablet for tablet eb9fa6fac8be4df5baa0b4a67e6890ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:22:33 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:47212
I20260808 18:22:33.112249 25606 raft_consensus.cc:2229] T eb9fa6fac8be4df5baa0b4a67e6890ce P 2a7890b8ca554c3192dcfefc94adcc48 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:22:33.112367 25606 raft_consensus.cc:2258] T eb9fa6fac8be4df5baa0b4a67e6890ce P 2a7890b8ca554c3192dcfefc94adcc48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:22:33.112449 25607 tablet_replica.cc:330] T eb9fa6fac8be4df5baa0b4a67e6890ce P a741b74692a642c697c799cdf11d6c27: stopping tablet replica
I20260808 18:22:33.112571 25607 raft_consensus.cc:2229] T eb9fa6fac8be4df5baa0b4a67e6890ce P a741b74692a642c697c799cdf11d6c27 [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:22:33.112869 25607 raft_consensus.cc:2258] T eb9fa6fac8be4df5baa0b4a67e6890ce P a741b74692a642c697c799cdf11d6c27 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:22:33.113718 25606 ts_tablet_manager.cc:1905] T eb9fa6fac8be4df5baa0b4a67e6890ce P 2a7890b8ca554c3192dcfefc94adcc48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:22:33.113744 25605 ts_tablet_manager.cc:1905] T eb9fa6fac8be4df5baa0b4a67e6890ce P 5553dc2bdc4248aca8e7b3e808a7a95e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:22:33.116123 25606 ts_tablet_manager.cc:1918] T eb9fa6fac8be4df5baa0b4a67e6890ce P 2a7890b8ca554c3192dcfefc94adcc48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 18:22:33.116179 25606 log.cc:1196] T eb9fa6fac8be4df5baa0b4a67e6890ce P 2a7890b8ca554c3192dcfefc94adcc48: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-2/wal/wals/eb9fa6fac8be4df5baa0b4a67e6890ce
I20260808 18:22:33.116367 25606 ts_tablet_manager.cc:1939] T eb9fa6fac8be4df5baa0b4a67e6890ce P 2a7890b8ca554c3192dcfefc94adcc48: Deleting consensus metadata
I20260808 18:22:33.116746 25605 ts_tablet_manager.cc:1918] T eb9fa6fac8be4df5baa0b4a67e6890ce P 5553dc2bdc4248aca8e7b3e808a7a95e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 18:22:33.116812 25605 log.cc:1196] T eb9fa6fac8be4df5baa0b4a67e6890ce P 5553dc2bdc4248aca8e7b3e808a7a95e: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-0/wal/wals/eb9fa6fac8be4df5baa0b4a67e6890ce
I20260808 18:22:33.117069 25605 ts_tablet_manager.cc:1939] T eb9fa6fac8be4df5baa0b4a67e6890ce P 5553dc2bdc4248aca8e7b3e808a7a95e: Deleting consensus metadata
I20260808 18:22:33.117144 25024 catalog_manager.cc:4841] TS 2a7890b8ca554c3192dcfefc94adcc48 (127.23.72.131:41455): tablet eb9fa6fac8be4df5baa0b4a67e6890ce (table test-table [id=b39f1b099fc241b6b3acd6f51c7b9157]) successfully deleted
I20260808 18:22:33.117842 25026 catalog_manager.cc:4841] TS 5553dc2bdc4248aca8e7b3e808a7a95e (127.23.72.129:41447): tablet eb9fa6fac8be4df5baa0b4a67e6890ce (table test-table [id=b39f1b099fc241b6b3acd6f51c7b9157]) successfully deleted
I20260808 18:22:33.118690 25607 ts_tablet_manager.cc:1905] T eb9fa6fac8be4df5baa0b4a67e6890ce P a741b74692a642c697c799cdf11d6c27: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:22:33.121495 25607 ts_tablet_manager.cc:1918] T eb9fa6fac8be4df5baa0b4a67e6890ce P a741b74692a642c697c799cdf11d6c27: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 18:22:33.121570 25607 log.cc:1196] T eb9fa6fac8be4df5baa0b4a67e6890ce P a741b74692a642c697c799cdf11d6c27: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213345431840-23842-0/minicluster-data/ts-1/wal/wals/eb9fa6fac8be4df5baa0b4a67e6890ce
I20260808 18:22:33.121822 25607 ts_tablet_manager.cc:1939] T eb9fa6fac8be4df5baa0b4a67e6890ce P a741b74692a642c697c799cdf11d6c27: Deleting consensus metadata
I20260808 18:22:33.122509 25029 catalog_manager.cc:4841] TS a741b74692a642c697c799cdf11d6c27 (127.23.72.130:36155): tablet eb9fa6fac8be4df5baa0b4a67e6890ce (table test-table [id=b39f1b099fc241b6b3acd6f51c7b9157]) successfully deleted
Aug 08 18:22:33 dist-test-slave-5k54 krb5kdc[24995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213352, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.23.72.129@KRBTEST.COM
I20260808 18:22:33.132470 25580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:33.117650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54054 (local address 127.23.72.129:41447)
0808 18:22:33.117793 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:33.117798 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:33.119851 (+ 2053us) server_negotiation.cc:407] Connection header received
0808 18:22:33.120054 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:33.120057 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:33.120105 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:33.120155 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:33.121207 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:33.121977 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:33.123376 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:33.123736 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:33.123862 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:33.127917 (+ 4055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:33.127929 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:33.127948 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:33.127978 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:33.130100 (+ 2122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:33.130666 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:33.130673 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:33.130674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:33.130770 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:33.131067 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:33.131070 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:33.131072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:33.131482 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:33.131687 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:33.131987 (+ 300us) server_negotiation.cc:299] Negotiation successful
0808 18:22:33.132115 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260808 18:22:33.133055 25250 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54054
I20260808 18:22:33.144688 25094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:33.134924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47434 (local address 127.23.72.190:40501)
0808 18:22:33.135056 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:33.135062 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:33.135554 (+ 492us) server_negotiation.cc:407] Connection header received
0808 18:22:33.135658 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:33.135662 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:33.135717 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:33.135831 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:33.136679 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:33.137519 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:33.138611 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:33.138963 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:33.139067 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:33.140832 (+ 1765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:33.140836 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:33.140839 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:33.140873 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:33.142772 (+ 1899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:33.143257 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:33.143261 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:33.143263 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:33.143360 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:33.143742 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:33.143745 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:33.143747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:33.143967 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:33.144095 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:33.144406 (+ 311us) server_negotiation.cc:299] Negotiation successful
0808 18:22:33.144523 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260808 18:22:33.145285 25040 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47434
I20260808 18:22:33.147173 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 25118
I20260808 18:22:33.158169 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 25265
I20260808 18:22:33.165387 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 25415
I20260808 18:22:33.171729 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 25008
2026-08-08T18:22:33Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2872 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:22:33 dist-test-slave-5k54 krb5kdc[25652](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:22:33 dist-test-slave-5k54 krb5kdc[25652](info): set up 2 sockets
Aug 08 18:22:33 dist-test-slave-5k54 krb5kdc[25652](info): commencing operation
krb5kdc: starting...
W20260808 18:22:34.281888 23842 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Aug 08 18:22:34 dist-test-slave-5k54 krb5kdc[25652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:22:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:22:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.72.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
I20260808 18:22:34.407840 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:45825
--webserver_interface=127.23.72.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:40327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:45825
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:34.519547 25667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:34.519872 25667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:34.519956 25667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:34.524062 25667 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:22:34.524148 25667 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 18:22:34.524179 25667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:34.524220 25667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:34.524277 25667 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:22:34.524310 25667 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:22:34.529611 25667 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:40327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:45825
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:45825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.72.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:34.530727 25667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:34.531845 25667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:34.540331 25676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:34.540422 25673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:34.540372 25674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:34.540560 25667 server_base.cc:1034] running on GCE node
I20260808 18:22:34.541280 25667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:34.542428 25667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:34.543597 25667 hybrid_clock.cc:648] HybridClock initialized: now 1786213354543567 us; error 44 us; skew 500 ppm
Aug 08 18:22:34 dist-test-slave-5k54 krb5kdc[25652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213354, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:34.551568 25667 init.cc:377] Logged in from keytab as kudu/127.23.72.190@KRBTEST.COM (short username kudu)
I20260808 18:22:34.552440 25667 webserver.cc:460] Webserver started at http://127.23.72.190:43233/ using document root <none> and password file <none>
I20260808 18:22:34.553015 25667 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:34.553112 25667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:34.553357 25667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:34.555410 25667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "9d816eb59738467ebf0d0d6cbf7bbcaf"
format_stamp: "Formatted at 2026-08-08 18:22:34 on dist-test-slave-5k54"
server_key: "16bec85dbb8f94d7d002fdb9bc2b138f"
server_key_iv: "4bdc97a29145250dd1bbcf80575984eb"
server_key_version: "encryptionkey@0"
I20260808 18:22:34.556105 25667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "9d816eb59738467ebf0d0d6cbf7bbcaf"
format_stamp: "Formatted at 2026-08-08 18:22:34 on dist-test-slave-5k54"
server_key: "16bec85dbb8f94d7d002fdb9bc2b138f"
server_key_iv: "4bdc97a29145250dd1bbcf80575984eb"
server_key_version: "encryptionkey@0"
I20260808 18:22:34.559892 25667 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 18:22:34.562402 25684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:34.563684 25667 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:22:34.563825 25667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "9d816eb59738467ebf0d0d6cbf7bbcaf"
format_stamp: "Formatted at 2026-08-08 18:22:34 on dist-test-slave-5k54"
server_key: "16bec85dbb8f94d7d002fdb9bc2b138f"
server_key_iv: "4bdc97a29145250dd1bbcf80575984eb"
server_key_version: "encryptionkey@0"
I20260808 18:22:34.563968 25667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:34.580034 25667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:34.583088 25667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:34.583297 25667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:34.591799 25667 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.190:45825
I20260808 18:22:34.591816 25750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.190:45825 every 8 connection(s)
I20260808 18:22:34.592779 25667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:22:34.594527 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 25667
I20260808 18:22:34.594625 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:22:34.594918 23842 external_mini_cluster.cc:1428] Setting key 3c94e27791a5befdfa28d793960139a5
I20260808 18:22:34.595916 25751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:34.602705 25751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d816eb59738467ebf0d0d6cbf7bbcaf: Bootstrap starting.
Aug 08 18:22:34 dist-test-slave-5k54 krb5kdc[25652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:22:34.605253 25751 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d816eb59738467ebf0d0d6cbf7bbcaf: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:34.606207 25751 log.cc:824] T 00000000000000000000000000000000 P 9d816eb59738467ebf0d0d6cbf7bbcaf: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:34.608618 25751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d816eb59738467ebf0d0d6cbf7bbcaf: No bootstrap required, opened a new log
I20260808 18:22:34.611186 25755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:34.596436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39076 (local address 127.23.72.190:45825)
0808 18:22:34.596866 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:34.596886 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:34.596930 (+ 44us) server_negotiation.cc:407] Connection header received
0808 18:22:34.597813 (+ 883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:34.597836 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:34.598251 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:34.598637 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:34.599406 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:34.600929 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:34.601835 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:34.602515 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:34.602676 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:34.605522 (+ 2846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:34.605529 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:34.605549 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:34.605585 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:34.608157 (+ 2572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:34.608591 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:34.608596 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:34.608601 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:34.608777 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:34.609054 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:34.609057 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:34.609059 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:34.609556 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:34.609758 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:34.609949 (+ 191us) server_negotiation.cc:299] Negotiation successful
0808 18:22:34.610245 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":114,"threads_started":1}
I20260808 18:22:34.612576 25751 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d816eb59738467ebf0d0d6cbf7bbcaf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d816eb59738467ebf0d0d6cbf7bbcaf" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 45825 } }
I20260808 18:22:34.612787 25751 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9d816eb59738467ebf0d0d6cbf7bbcaf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:34.612846 25751 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d816eb59738467ebf0d0d6cbf7bbcaf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d816eb59738467ebf0d0d6cbf7bbcaf, State: Initialized, Role: FOLLOWER
I20260808 18:22:34.613257 25751 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d816eb59738467ebf0d0d6cbf7bbcaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d816eb59738467ebf0d0d6cbf7bbcaf" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 45825 } }
I20260808 18:22:34.613433 25751 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d816eb59738467ebf0d0d6cbf7bbcaf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:22:34.613494 25751 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d816eb59738467ebf0d0d6cbf7bbcaf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:22:34.613586 25751 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d816eb59738467ebf0d0d6cbf7bbcaf [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:34.614482 25751 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d816eb59738467ebf0d0d6cbf7bbcaf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d816eb59738467ebf0d0d6cbf7bbcaf" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 45825 } }
I20260808 18:22:34.614818 25751 leader_election.cc:302] T 00000000000000000000000000000000 P 9d816eb59738467ebf0d0d6cbf7bbcaf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d816eb59738467ebf0d0d6cbf7bbcaf; no voters:
I20260808 18:22:34.615160 25751 leader_election.cc:288] T 00000000000000000000000000000000 P 9d816eb59738467ebf0d0d6cbf7bbcaf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:22:34.615273 25757 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d816eb59738467ebf0d0d6cbf7bbcaf [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:22:34.615545 25757 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d816eb59738467ebf0d0d6cbf7bbcaf [term 1 LEADER]: Becoming Leader. State: Replica: 9d816eb59738467ebf0d0d6cbf7bbcaf, State: Running, Role: LEADER
I20260808 18:22:34.615844 25757 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d816eb59738467ebf0d0d6cbf7bbcaf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d816eb59738467ebf0d0d6cbf7bbcaf" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 45825 } }
I20260808 18:22:34.616355 25751 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d816eb59738467ebf0d0d6cbf7bbcaf [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:22:34.617003 25759 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d816eb59738467ebf0d0d6cbf7bbcaf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d816eb59738467ebf0d0d6cbf7bbcaf. Latest consensus state: current_term: 1 leader_uuid: "9d816eb59738467ebf0d0d6cbf7bbcaf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d816eb59738467ebf0d0d6cbf7bbcaf" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 45825 } } }
I20260808 18:22:34.617169 25759 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d816eb59738467ebf0d0d6cbf7bbcaf [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:34.617519 25758 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d816eb59738467ebf0d0d6cbf7bbcaf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9d816eb59738467ebf0d0d6cbf7bbcaf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d816eb59738467ebf0d0d6cbf7bbcaf" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 45825 } } }
I20260808 18:22:34.617624 25758 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d816eb59738467ebf0d0d6cbf7bbcaf [sys.catalog]: This master's current role is: LEADER
W20260808 18:22:34.621381 25775 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9d816eb59738467ebf0d0d6cbf7bbcaf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:22:34.621448 25775 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:22:34.621528 25769 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:22:34.622226 25769 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:22:34.627636 25769 catalog_manager.cc:1347] Generated new cluster ID: 151be29ab19d4f3b9f2cb9ba65d797aa
I20260808 18:22:34.627698 25769 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:22:34.634836 25769 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:22:34.635787 25769 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:22:34.652755 25769 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d816eb59738467ebf0d0d6cbf7bbcaf: Generated new TSK 0
I20260808 18:22:34.653535 25769 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.72.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab.
Entry for principal kudu/127.23.72.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab.
I20260808 18:22:34.688515 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.129:0
--local_ip_for_outbound_sockets=127.23.72.129
--webserver_interface=127.23.72.129
--webserver_port=0
--tserver_master_addrs=127.23.72.190:45825
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab
--principal=kudu/127.23.72.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:40327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:34.801841 25781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:34.802078 25781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:34.802158 25781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:34.806277 25781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:34.806334 25781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:34.806408 25781 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 18:22:34.806440 25781 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 18:22:34.806529 25781 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.129
I20260808 18:22:34.811589 25781 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:40327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab
--principal=kudu/127.23.72.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.72.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.72.190:45825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25781
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.129
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:34.812613 25781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:34.813632 25781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:34.823607 25789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:34.823649 25787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:34.823705 25781 server_base.cc:1034] running on GCE node
W20260808 18:22:34.823637 25786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:34.824232 25781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:34.824885 25781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:34.826054 25781 hybrid_clock.cc:648] HybridClock initialized: now 1786213354826040 us; error 24 us; skew 500 ppm
Aug 08 18:22:34 dist-test-slave-5k54 krb5kdc[25652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213354, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:34.828933 25781 init.cc:377] Logged in from keytab as kudu/127.23.72.129@KRBTEST.COM (short username kudu)
I20260808 18:22:34.829746 25781 webserver.cc:460] Webserver started at http://127.23.72.129:39843/ using document root <none> and password file <none>
I20260808 18:22:34.830257 25781 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:34.830317 25781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:34.830488 25781 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:34.832541 25781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-0/data/instance:
uuid: "8d738d4e749a48be988ae86c964d91c8"
format_stamp: "Formatted at 2026-08-08 18:22:34 on dist-test-slave-5k54"
server_key: "01d25041ecf2807b71aa142c1f540d99"
server_key_iv: "14b3a5c652aaa9d7070d9049dcdb5b6d"
server_key_version: "encryptionkey@0"
I20260808 18:22:34.833071 25781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance:
uuid: "8d738d4e749a48be988ae86c964d91c8"
format_stamp: "Formatted at 2026-08-08 18:22:34 on dist-test-slave-5k54"
server_key: "01d25041ecf2807b71aa142c1f540d99"
server_key_iv: "14b3a5c652aaa9d7070d9049dcdb5b6d"
server_key_version: "encryptionkey@0"
I20260808 18:22:34.837049 25781 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260808 18:22:34.839365 25796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:34.840623 25781 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 18:22:34.840744 25781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-0/wal
uuid: "8d738d4e749a48be988ae86c964d91c8"
format_stamp: "Formatted at 2026-08-08 18:22:34 on dist-test-slave-5k54"
server_key: "01d25041ecf2807b71aa142c1f540d99"
server_key_iv: "14b3a5c652aaa9d7070d9049dcdb5b6d"
server_key_version: "encryptionkey@0"
I20260808 18:22:34.840852 25781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:34.863910 25781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:34.866812 25781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:34.867048 25781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:34.868613 25781 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:22:34.868660 25781 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:34.868701 25781 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:22:34.868753 25781 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 18:22:34 dist-test-slave-5k54 krb5kdc[25652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213354, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:22:34.880899 25781 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.129:33739
I20260808 18:22:34.880998 25922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.129:33739 every 8 connection(s)
I20260808 18:22:34.882047 25781 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
I20260808 18:22:34.884991 25755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:34.869652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:54645 (local address 127.23.72.190:45825)
0808 18:22:34.869884 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:34.869889 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:34.871006 (+ 1117us) server_negotiation.cc:407] Connection header received
0808 18:22:34.872534 (+ 1528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:34.872540 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:34.872609 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:34.872675 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:34.874544 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:34.875343 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:34.877092 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:34.877490 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:34.877569 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:34.881134 (+ 3565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:34.881138 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:34.881141 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:34.881169 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:34.882865 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:34.883364 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:34.883368 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:34.883370 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:34.883493 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:34.883868 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:34.883871 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:34.883873 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:34.884062 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:34.884166 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:34.884658 (+ 492us) server_negotiation.cc:299] Negotiation successful
0808 18:22:34.884808 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
I20260808 18:22:34.885728 25805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:34.870171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:45825 (local address 127.23.72.129:54645)
0808 18:22:34.870811 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:34.870886 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:34.872257 (+ 1371us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:34.873124 (+ 867us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:34.873134 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:34.873672 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:34.874343 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:34.874363 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:34.875540 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:34.875547 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:34.876940 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:34.876955 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:34.877788 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:34.877792 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:34.878013 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:34.878708 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:34.878729 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:34.880908 (+ 2179us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:34.882973 (+ 2065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:34.882980 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:34.882994 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:34.883250 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:34.883605 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:34.883608 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:34.883610 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:34.883757 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:34.884179 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:34.884187 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:34.884424 (+ 237us) client_negotiation.cc:769] Sending connection context
0808 18:22:34.884598 (+ 174us) client_negotiation.cc:240] Negotiation successful
0808 18:22:34.884926 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":179,"threads_started":1}
I20260808 18:22:34.886987 25923 heartbeater.cc:347] Connected to a master server at 127.23.72.190:45825
I20260808 18:22:34.887151 25923 heartbeater.cc:464] Registering TS with master...
I20260808 18:22:34.887684 25923 heartbeater.cc:511] Master 127.23.72.190:45825 requested a full tablet report, sending...
I20260808 18:22:34.889104 25703 ts_manager.cc:194] Registered new tserver with Master: 8d738d4e749a48be988ae86c964d91c8 (127.23.72.129:33739)
I20260808 18:22:34.889456 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 25781
I20260808 18:22:34.889569 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance
I20260808 18:22:34.889863 23842 external_mini_cluster.cc:1428] Setting key 2bf87a6bc6d8aa515b803e06357e27b3
I20260808 18:22:34.890641 25703 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.129@KRBTEST.COM'} at 127.23.72.129:54645
I20260808 18:22:34.897377 25929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:34.889211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:45825 (local address 127.23.72.129:33851)
0808 18:22:34.889507 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:34.889518 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:34.889624 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:34.889854 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:34.889857 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:34.890061 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:34.890151 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:34.890156 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:34.891184 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:34.891187 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:34.892099 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:34.892113 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:34.892685 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:34.892692 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:34.892815 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:34.893304 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:34.893323 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:34.893792 (+ 469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:34.895847 (+ 2055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:34.895850 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:34.895851 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:34.896052 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:34.896369 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:34.896371 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:34.896372 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:34.896508 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:34.896971 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:34.896974 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:34.897047 (+ 73us) client_negotiation.cc:769] Sending connection context
0808 18:22:34.897127 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 18:22:34.897228 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":85,"threads_started":1}
I20260808 18:22:34.897739 25755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:34.889258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:33851 (local address 127.23.72.190:45825)
0808 18:22:34.889379 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:34.889383 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:34.889588 (+ 205us) server_negotiation.cc:407] Connection header received
0808 18:22:34.889698 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:34.889701 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:34.889747 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:34.889830 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:34.890307 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:34.891011 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:34.892236 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:34.892554 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:34.892663 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:34.893962 (+ 1299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:34.893966 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:34.893969 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:34.893992 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:34.895740 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:34.896159 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:34.896166 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:34.896171 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:34.896272 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:34.896613 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:34.896618 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:34.896623 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:34.896876 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:34.896982 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:34.897220 (+ 238us) server_negotiation.cc:299] Negotiation successful
0808 18:22:34.897355 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":48}
WARNING: no policy specified for kudu/127.23.72.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab.
Entry for principal kudu/127.23.72.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab.
I20260808 18:22:34.919220 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.130:0
--local_ip_for_outbound_sockets=127.23.72.130
--webserver_interface=127.23.72.130
--webserver_port=0
--tserver_master_addrs=127.23.72.190:45825
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab
--principal=kudu/127.23.72.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:40327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:35.030969 25932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:35.031251 25932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:35.031330 25932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:35.035360 25932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:35.035460 25932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:35.035538 25932 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 18:22:35.035579 25932 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 18:22:35.035640 25932 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.130
I20260808 18:22:35.040856 25932 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:40327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab
--principal=kudu/127.23.72.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.72.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.72.190:45825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25932
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.130
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:35.041898 25932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:35.042910 25932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:35.052461 25941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:35.052485 25938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:35.052413 25939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:35.052613 25932 server_base.cc:1034] running on GCE node
I20260808 18:22:35.053046 25932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:35.053699 25932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:35.054855 25932 hybrid_clock.cc:648] HybridClock initialized: now 1786213355054838 us; error 30 us; skew 500 ppm
Aug 08 18:22:35 dist-test-slave-5k54 krb5kdc[25652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213355, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:35.057667 25932 init.cc:377] Logged in from keytab as kudu/127.23.72.130@KRBTEST.COM (short username kudu)
I20260808 18:22:35.058497 25932 webserver.cc:460] Webserver started at http://127.23.72.130:33281/ using document root <none> and password file <none>
I20260808 18:22:35.059083 25932 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:35.059181 25932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:35.059476 25932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:35.061524 25932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-1/data/instance:
uuid: "a4ced4ae4cb143ec86ef7b0a99711d22"
format_stamp: "Formatted at 2026-08-08 18:22:35 on dist-test-slave-5k54"
server_key: "84d26bd75b098c51754f6037f9e35c18"
server_key_iv: "107a6deb831df147271ed93d91600777"
server_key_version: "encryptionkey@0"
I20260808 18:22:35.062093 25932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance:
uuid: "a4ced4ae4cb143ec86ef7b0a99711d22"
format_stamp: "Formatted at 2026-08-08 18:22:35 on dist-test-slave-5k54"
server_key: "84d26bd75b098c51754f6037f9e35c18"
server_key_iv: "107a6deb831df147271ed93d91600777"
server_key_version: "encryptionkey@0"
I20260808 18:22:35.065789 25932 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260808 18:22:35.068377 25948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:35.069597 25932 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:22:35.069725 25932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-1/wal
uuid: "a4ced4ae4cb143ec86ef7b0a99711d22"
format_stamp: "Formatted at 2026-08-08 18:22:35 on dist-test-slave-5k54"
server_key: "84d26bd75b098c51754f6037f9e35c18"
server_key_iv: "107a6deb831df147271ed93d91600777"
server_key_version: "encryptionkey@0"
I20260808 18:22:35.069846 25932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:35.078919 25932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:35.082334 25932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:35.082538 25932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:35.084053 25932 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:22:35.084124 25932 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:35.084196 25932 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:22:35.084245 25932 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 18:22:35 dist-test-slave-5k54 krb5kdc[25652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213355, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.130@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:22:35.095405 25932 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.130:36341
I20260808 18:22:35.095643 26071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.130:36341 every 8 connection(s)
I20260808 18:22:35.096342 25932 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
I20260808 18:22:35.098388 25755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:35.084741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:38817 (local address 127.23.72.190:45825)
0808 18:22:35.084936 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:35.084943 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:35.086440 (+ 1497us) server_negotiation.cc:407] Connection header received
0808 18:22:35.087960 (+ 1520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:35.087964 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:35.088027 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:35.088149 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:35.089432 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.089971 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.091446 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.091674 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.091790 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:35.094458 (+ 2668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:35.094462 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:35.094464 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:35.094488 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:35.096168 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:35.096711 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:35.096716 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:35.096718 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:35.096827 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:35.097275 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:35.097279 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:35.097281 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:35.097474 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:35.097579 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:35.098076 (+ 497us) server_negotiation.cc:299] Negotiation successful
0808 18:22:35.098199 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 18:22:35.099123 25958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:35.085695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:45825 (local address 127.23.72.130:38817)
0808 18:22:35.086265 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:35.086322 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:35.087717 (+ 1395us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:35.088371 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:35.088377 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:35.088701 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:35.089266 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:35.089284 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.090096 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.090100 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:35.091293 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:35.091303 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.091840 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.091844 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:35.092028 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:35.092655 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:35.092670 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:35.094214 (+ 1544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:35.096278 (+ 2064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:35.096284 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:35.096297 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:35.096610 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:35.097001 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:35.097004 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:35.097006 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:35.097170 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:35.097592 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:35.097600 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:35.097839 (+ 239us) client_negotiation.cc:769] Sending connection context
0808 18:22:35.098028 (+ 189us) client_negotiation.cc:240] Negotiation successful
0808 18:22:35.098333 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":151,"threads_started":1}
I20260808 18:22:35.100607 26072 heartbeater.cc:347] Connected to a master server at 127.23.72.190:45825
I20260808 18:22:35.100769 26072 heartbeater.cc:464] Registering TS with master...
I20260808 18:22:35.101279 26072 heartbeater.cc:511] Master 127.23.72.190:45825 requested a full tablet report, sending...
I20260808 18:22:35.102418 25704 ts_manager.cc:194] Registered new tserver with Master: a4ced4ae4cb143ec86ef7b0a99711d22 (127.23.72.130:36341)
I20260808 18:22:35.103183 25704 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.130@KRBTEST.COM'} at 127.23.72.130:38817
I20260808 18:22:35.105742 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 25932
I20260808 18:22:35.105888 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance
I20260808 18:22:35.106222 23842 external_mini_cluster.cc:1428] Setting key aef841fd7123a67b5f654a1dd3c97632
I20260808 18:22:35.110605 25755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:35.103025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:37993 (local address 127.23.72.190:45825)
0808 18:22:35.103144 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:35.103148 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:35.103295 (+ 147us) server_negotiation.cc:407] Connection header received
0808 18:22:35.103413 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:35.103415 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:35.103515 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:35.103595 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:35.104102 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.104874 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.105828 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.106123 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.106204 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:35.107494 (+ 1290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:35.107498 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:35.107500 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:35.107530 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:35.108881 (+ 1351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:35.109302 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:35.109306 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:35.109314 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:35.109397 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:35.109693 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:35.109696 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:35.109698 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:35.109916 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:35.110052 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:35.110329 (+ 277us) server_negotiation.cc:299] Negotiation successful
0808 18:22:35.110443 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 18:22:35.110613 26078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:35.102887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:45825 (local address 127.23.72.130:37993)
0808 18:22:35.103190 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:35.103208 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:35.103315 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:35.103634 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:35.103637 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:35.103865 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:35.103985 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:35.103991 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.105019 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.105022 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:35.105686 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:35.105698 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.106236 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.106241 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:35.106338 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:35.106901 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:35.106913 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:35.107327 (+ 414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:35.109002 (+ 1675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:35.109006 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:35.109008 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:35.109204 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:35.109505 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:35.109508 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:35.109510 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:35.109595 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:35.110039 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:35.110042 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:35.110156 (+ 114us) client_negotiation.cc:769] Sending connection context
0808 18:22:35.110257 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 18:22:35.110400 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":98,"threads_started":1}
WARNING: no policy specified for kudu/127.23.72.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab.
Entry for principal kudu/127.23.72.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab.
I20260808 18:22:35.135067 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.131:0
--local_ip_for_outbound_sockets=127.23.72.131
--webserver_interface=127.23.72.131
--webserver_port=0
--tserver_master_addrs=127.23.72.190:45825
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab
--principal=kudu/127.23.72.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:40327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:35.248723 26081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:35.248970 26081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:35.249014 26081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:35.253028 26081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:35.253083 26081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:35.253122 26081 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 18:22:35.253136 26081 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 18:22:35.253167 26081 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.131
I20260808 18:22:35.258200 26081 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:40327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab
--principal=kudu/127.23.72.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.72.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.72.190:45825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26081
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.131
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:35.259357 26081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:35.260396 26081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:35.270072 26087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:35.270095 26089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:35.270095 26086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:35.270417 26081 server_base.cc:1034] running on GCE node
I20260808 18:22:35.270809 26081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:35.271484 26081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:35.272688 26081 hybrid_clock.cc:648] HybridClock initialized: now 1786213355272650 us; error 59 us; skew 500 ppm
Aug 08 18:22:35 dist-test-slave-5k54 krb5kdc[25652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213355, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:35.275658 26081 init.cc:377] Logged in from keytab as kudu/127.23.72.131@KRBTEST.COM (short username kudu)
I20260808 18:22:35.276492 26081 webserver.cc:460] Webserver started at http://127.23.72.131:42559/ using document root <none> and password file <none>
I20260808 18:22:35.277083 26081 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:35.277179 26081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:35.277410 26081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:35.279469 26081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-2/data/instance:
uuid: "6bd92cb899f44c2cbe8bc5be5655f22f"
format_stamp: "Formatted at 2026-08-08 18:22:35 on dist-test-slave-5k54"
server_key: "e726108be16761fd28d64990c090353a"
server_key_iv: "1e85aaff0a232775fbe5118d5519bc93"
server_key_version: "encryptionkey@0"
I20260808 18:22:35.280053 26081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance:
uuid: "6bd92cb899f44c2cbe8bc5be5655f22f"
format_stamp: "Formatted at 2026-08-08 18:22:35 on dist-test-slave-5k54"
server_key: "e726108be16761fd28d64990c090353a"
server_key_iv: "1e85aaff0a232775fbe5118d5519bc93"
server_key_version: "encryptionkey@0"
I20260808 18:22:35.283865 26081 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260808 18:22:35.286214 26098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:35.287590 26081 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:22:35.287683 26081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-2/wal
uuid: "6bd92cb899f44c2cbe8bc5be5655f22f"
format_stamp: "Formatted at 2026-08-08 18:22:35 on dist-test-slave-5k54"
server_key: "e726108be16761fd28d64990c090353a"
server_key_iv: "1e85aaff0a232775fbe5118d5519bc93"
server_key_version: "encryptionkey@0"
I20260808 18:22:35.287770 26081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:35.301446 26081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:35.304647 26081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:35.304818 26081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:35.306217 26081 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:22:35.306294 26081 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:35.306366 26081 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:22:35.306419 26081 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:35.316773 26081 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.131:40149
I20260808 18:22:35.316848 26222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.131:40149 every 8 connection(s)
Aug 08 18:22:35 dist-test-slave-5k54 krb5kdc[25652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213355, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.131@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:22:35.317863 26081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
I20260808 18:22:35.321326 25755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:35.307271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:33525 (local address 127.23.72.190:45825)
0808 18:22:35.307478 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:35.307487 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:35.308499 (+ 1012us) server_negotiation.cc:407] Connection header received
0808 18:22:35.309753 (+ 1254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:35.309757 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:35.309829 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:35.309931 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:35.311699 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.312609 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.313984 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.314361 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.314486 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:35.317645 (+ 3159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:35.317649 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:35.317652 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:35.317682 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:35.319350 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:35.319876 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:35.319879 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:35.319880 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:35.319955 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:35.320246 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:35.320248 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:35.320249 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:35.320414 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:35.320484 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:35.321098 (+ 614us) server_negotiation.cc:299] Negotiation successful
0808 18:22:35.321203 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 18:22:35.321836 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 26081
I20260808 18:22:35.321974 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance
I20260808 18:22:35.322264 23842 external_mini_cluster.cc:1428] Setting key cd0c3aa1cb4d4bd702fc63baeaba1f10
I20260808 18:22:35.322291 26107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:35.307823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:45825 (local address 127.23.72.131:33525)
0808 18:22:35.308337 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:35.308378 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:35.309482 (+ 1104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:35.310144 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:35.310151 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:35.310690 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:35.311520 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:35.311542 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.312746 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.312750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:35.313848 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:35.313858 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.314578 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.314584 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:35.314852 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:35.315518 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:35.315541 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:35.317479 (+ 1938us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:35.319486 (+ 2007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:35.319491 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:35.319503 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:35.319751 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:35.320035 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:35.320037 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:35.320039 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:35.320163 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:35.320539 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:35.320545 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:35.320773 (+ 228us) client_negotiation.cc:769] Sending connection context
0808 18:22:35.321003 (+ 230us) client_negotiation.cc:240] Negotiation successful
0808 18:22:35.321273 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":129,"threads_started":1}
I20260808 18:22:35.324411 26223 heartbeater.cc:347] Connected to a master server at 127.23.72.190:45825
I20260808 18:22:35.324574 26223 heartbeater.cc:464] Registering TS with master...
I20260808 18:22:35.325115 26223 heartbeater.cc:511] Master 127.23.72.190:45825 requested a full tablet report, sending...
I20260808 18:22:35.326295 25703 ts_manager.cc:194] Registered new tserver with Master: 6bd92cb899f44c2cbe8bc5be5655f22f (127.23.72.131:40149)
I20260808 18:22:35.326915 25703 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.131@KRBTEST.COM'} at 127.23.72.131:33525
I20260808 18:22:35.332664 25755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:35.325923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:51085 (local address 127.23.72.190:45825)
0808 18:22:35.326068 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:35.326073 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:35.326253 (+ 180us) server_negotiation.cc:407] Connection header received
0808 18:22:35.326362 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:35.326365 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:35.326413 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:35.326484 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:35.326898 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.327614 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.328456 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.328685 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.328791 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:35.329812 (+ 1021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:35.329815 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:35.329818 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:35.329842 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:35.331057 (+ 1215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:35.331504 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:35.331508 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:35.331509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:35.331593 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:35.331873 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:35.331876 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:35.331878 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:35.332047 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:35.332152 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:35.332402 (+ 250us) server_negotiation.cc:299] Negotiation successful
0808 18:22:35.332507 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 18:22:35.332675 26230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:35.325872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:45825 (local address 127.23.72.131:51085)
0808 18:22:35.326176 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:35.326188 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:35.326291 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:35.326497 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:35.326500 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:35.326698 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:35.326783 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:35.326788 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.327734 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.327737 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:35.328324 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:35.328331 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.328795 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.328799 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:35.328890 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:35.329343 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:35.329374 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:35.329683 (+ 309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:35.331169 (+ 1486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:35.331174 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:35.331176 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:35.331375 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:35.331691 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:35.331693 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:35.331695 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:35.331779 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:35.332142 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:35.332145 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:35.332239 (+ 94us) client_negotiation.cc:769] Sending connection context
0808 18:22:35.332344 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 18:22:35.332488 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"mutex_wait_us":1,"thread_start_us":126,"threads_started":1}
I20260808 18:22:35.336619 23842 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 18:22:35 dist-test-slave-5k54 krb5kdc[25652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 18:22:35 dist-test-slave-5k54 krb5kdc[25652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:22:35.365989 25755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:35.357274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39092 (local address 127.23.72.190:45825)
0808 18:22:35.357429 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:35.357433 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:35.357446 (+ 13us) server_negotiation.cc:407] Connection header received
0808 18:22:35.357490 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:35.357493 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:35.357535 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:35.357604 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:35.358404 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.358971 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.360023 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.360255 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.360342 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:35.363002 (+ 2660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:35.363006 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:35.363008 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:35.363033 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:35.364480 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:35.364928 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:35.364931 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:35.364932 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:35.365008 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:35.365255 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:35.365257 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:35.365259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:35.365449 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:35.365563 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:35.365759 (+ 196us) server_negotiation.cc:299] Negotiation successful
0808 18:22:35.365856 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 18:22:35.369161 25703 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39092:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:22:35.372145 25703 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:22:35.386471 26244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:35.380466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46058 (local address 127.23.72.129:33739)
0808 18:22:35.381337 (+ 871us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:35.381353 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:35.381378 (+ 25us) server_negotiation.cc:407] Connection header received
0808 18:22:35.381457 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:35.381462 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:35.381672 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:35.381840 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:22:35.382119 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.383262 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.384966 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.385796 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.385915 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:35.386099 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:35.386218 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 18:22:35.386320 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":754,"thread_start_us":107,"threads_started":1}
I20260808 18:22:35.386519 26242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:35.380323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:33739 (local address 127.0.0.1:46058)
0808 18:22:35.380779 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:35.380831 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:35.380960 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:35.381823 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:35.381829 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:35.381852 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:22:35.381996 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:35.382003 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.383450 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.383453 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:35.384823 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:35.384836 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.385925 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.385929 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:35.386032 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:35.386075 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 18:22:35.386188 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 18:22:35.386326 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"mutex_wait_us":6,"thread_start_us":89,"threads_started":1}
I20260808 18:22:35.386581 26243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:35.380433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.130:36341 (local address 127.0.0.1:52570)
0808 18:22:35.380862 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:35.380876 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:35.380960 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:35.382618 (+ 1658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:35.382621 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:35.382639 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:22:35.382755 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:35.382765 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.383944 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.383949 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:35.385091 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:35.385101 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.386168 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.386173 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:35.386279 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:35.386305 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 18:22:35.386355 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 18:22:35.386420 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"mutex_wait_us":32,"thread_start_us":83,"threads_started":1}
I20260808 18:22:35.387104 26245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:35.381081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.131:40149 (local address 127.0.0.1:35138)
0808 18:22:35.381337 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:35.381352 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:35.381443 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:35.382789 (+ 1346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:35.382794 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:35.382814 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:22:35.382923 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:35.382933 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.384333 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.384336 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:35.385401 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:35.385408 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.386689 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.386693 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:35.386799 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:35.386824 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 18:22:35.386901 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 18:22:35.386991 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":5,"thread_start_us":118,"threads_started":1}
I20260808 18:22:35.387250 26247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:35.381542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52570 (local address 127.23.72.130:36341)
0808 18:22:35.382211 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:35.382223 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:35.382243 (+ 20us) server_negotiation.cc:407] Connection header received
0808 18:22:35.382318 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:35.382323 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:35.382488 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:35.382641 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:22:35.382866 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.383829 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.385221 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.386023 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.386770 (+ 747us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:35.386920 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:35.387000 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 18:22:35.387098 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":125,"threads_started":1}
I20260808 18:22:35.387539 26246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:35.381214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35138 (local address 127.23.72.131:40149)
0808 18:22:35.382267 (+ 1053us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:35.382280 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:35.382302 (+ 22us) server_negotiation.cc:407] Connection header received
0808 18:22:35.382386 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:35.382391 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:35.382574 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:35.382711 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:22:35.383050 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.384212 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.385516 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.386370 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.386471 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:35.386625 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:35.387288 (+ 663us) server_negotiation.cc:299] Negotiation successful
0808 18:22:35.387383 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":132,"threads_started":1}
I20260808 18:22:35.389922 26153 tablet_service.cc:1467] Processing CreateTablet for tablet addbf42ee53347faa75e1747512c6b01 (DEFAULT_TABLE table=test-table [id=1083a8e507d54a6c9da884edb511b98c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:22:35.390062 25849 tablet_service.cc:1467] Processing CreateTablet for tablet addbf42ee53347faa75e1747512c6b01 (DEFAULT_TABLE table=test-table [id=1083a8e507d54a6c9da884edb511b98c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:22:35.390065 26001 tablet_service.cc:1467] Processing CreateTablet for tablet addbf42ee53347faa75e1747512c6b01 (DEFAULT_TABLE table=test-table [id=1083a8e507d54a6c9da884edb511b98c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:22:35.390935 26153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet addbf42ee53347faa75e1747512c6b01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:35.391044 25849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet addbf42ee53347faa75e1747512c6b01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:35.391044 26001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet addbf42ee53347faa75e1747512c6b01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:35.397744 26248 tablet_bootstrap.cc:492] T addbf42ee53347faa75e1747512c6b01 P 6bd92cb899f44c2cbe8bc5be5655f22f: Bootstrap starting.
I20260808 18:22:35.398648 26249 tablet_bootstrap.cc:492] T addbf42ee53347faa75e1747512c6b01 P a4ced4ae4cb143ec86ef7b0a99711d22: Bootstrap starting.
I20260808 18:22:35.399181 26250 tablet_bootstrap.cc:492] T addbf42ee53347faa75e1747512c6b01 P 8d738d4e749a48be988ae86c964d91c8: Bootstrap starting.
I20260808 18:22:35.400224 26248 tablet_bootstrap.cc:654] T addbf42ee53347faa75e1747512c6b01 P 6bd92cb899f44c2cbe8bc5be5655f22f: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:35.400828 26249 tablet_bootstrap.cc:654] T addbf42ee53347faa75e1747512c6b01 P a4ced4ae4cb143ec86ef7b0a99711d22: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:35.401114 26250 tablet_bootstrap.cc:654] T addbf42ee53347faa75e1747512c6b01 P 8d738d4e749a48be988ae86c964d91c8: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:35.401170 26248 log.cc:824] T addbf42ee53347faa75e1747512c6b01 P 6bd92cb899f44c2cbe8bc5be5655f22f: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:35.401659 26249 log.cc:824] T addbf42ee53347faa75e1747512c6b01 P a4ced4ae4cb143ec86ef7b0a99711d22: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:35.402345 26250 log.cc:824] T addbf42ee53347faa75e1747512c6b01 P 8d738d4e749a48be988ae86c964d91c8: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:35.403656 26249 tablet_bootstrap.cc:492] T addbf42ee53347faa75e1747512c6b01 P a4ced4ae4cb143ec86ef7b0a99711d22: No bootstrap required, opened a new log
I20260808 18:22:35.403687 26248 tablet_bootstrap.cc:492] T addbf42ee53347faa75e1747512c6b01 P 6bd92cb899f44c2cbe8bc5be5655f22f: No bootstrap required, opened a new log
I20260808 18:22:35.403861 26249 ts_tablet_manager.cc:1397] T addbf42ee53347faa75e1747512c6b01 P a4ced4ae4cb143ec86ef7b0a99711d22: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260808 18:22:35.403860 26248 ts_tablet_manager.cc:1397] T addbf42ee53347faa75e1747512c6b01 P 6bd92cb899f44c2cbe8bc5be5655f22f: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 18:22:35.404268 26250 tablet_bootstrap.cc:492] T addbf42ee53347faa75e1747512c6b01 P 8d738d4e749a48be988ae86c964d91c8: No bootstrap required, opened a new log
I20260808 18:22:35.404439 26250 ts_tablet_manager.cc:1397] T addbf42ee53347faa75e1747512c6b01 P 8d738d4e749a48be988ae86c964d91c8: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 18:22:35.406736 26249 raft_consensus.cc:348] T addbf42ee53347faa75e1747512c6b01 P a4ced4ae4cb143ec86ef7b0a99711d22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } }
I20260808 18:22:35.407006 26249 raft_consensus.cc:374] T addbf42ee53347faa75e1747512c6b01 P a4ced4ae4cb143ec86ef7b0a99711d22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:35.407089 26249 raft_consensus.cc:729] T addbf42ee53347faa75e1747512c6b01 P a4ced4ae4cb143ec86ef7b0a99711d22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4ced4ae4cb143ec86ef7b0a99711d22, State: Initialized, Role: FOLLOWER
I20260808 18:22:35.407558 26249 consensus_queue.cc:260] T addbf42ee53347faa75e1747512c6b01 P a4ced4ae4cb143ec86ef7b0a99711d22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } }
I20260808 18:22:35.407784 26248 raft_consensus.cc:348] T addbf42ee53347faa75e1747512c6b01 P 6bd92cb899f44c2cbe8bc5be5655f22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } }
I20260808 18:22:35.408224 26072 heartbeater.cc:503] Master 127.23.72.190:45825 was elected leader, sending a full tablet report...
I20260808 18:22:35.408118 26250 raft_consensus.cc:348] T addbf42ee53347faa75e1747512c6b01 P 8d738d4e749a48be988ae86c964d91c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } }
I20260808 18:22:35.408337 26250 raft_consensus.cc:374] T addbf42ee53347faa75e1747512c6b01 P 8d738d4e749a48be988ae86c964d91c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:35.408382 26250 raft_consensus.cc:729] T addbf42ee53347faa75e1747512c6b01 P 8d738d4e749a48be988ae86c964d91c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d738d4e749a48be988ae86c964d91c8, State: Initialized, Role: FOLLOWER
I20260808 18:22:35.408455 26249 ts_tablet_manager.cc:1428] T addbf42ee53347faa75e1747512c6b01 P a4ced4ae4cb143ec86ef7b0a99711d22: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260808 18:22:35.408738 26248 raft_consensus.cc:374] T addbf42ee53347faa75e1747512c6b01 P 6bd92cb899f44c2cbe8bc5be5655f22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:35.408826 26248 raft_consensus.cc:729] T addbf42ee53347faa75e1747512c6b01 P 6bd92cb899f44c2cbe8bc5be5655f22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bd92cb899f44c2cbe8bc5be5655f22f, State: Initialized, Role: FOLLOWER
I20260808 18:22:35.408843 26250 consensus_queue.cc:260] T addbf42ee53347faa75e1747512c6b01 P 8d738d4e749a48be988ae86c964d91c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } }
I20260808 18:22:35.409201 26248 consensus_queue.cc:260] T addbf42ee53347faa75e1747512c6b01 P 6bd92cb899f44c2cbe8bc5be5655f22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } }
I20260808 18:22:35.409436 25923 heartbeater.cc:503] Master 127.23.72.190:45825 was elected leader, sending a full tablet report...
I20260808 18:22:35.410140 26248 ts_tablet_manager.cc:1428] T addbf42ee53347faa75e1747512c6b01 P 6bd92cb899f44c2cbe8bc5be5655f22f: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 18:22:35.410357 26223 heartbeater.cc:503] Master 127.23.72.190:45825 was elected leader, sending a full tablet report...
I20260808 18:22:35.411269 26250 ts_tablet_manager.cc:1428] T addbf42ee53347faa75e1747512c6b01 P 8d738d4e749a48be988ae86c964d91c8: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260808 18:22:35.457926 26254 raft_consensus.cc:482] T addbf42ee53347faa75e1747512c6b01 P a4ced4ae4cb143ec86ef7b0a99711d22 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:22:35.458258 26254 raft_consensus.cc:504] T addbf42ee53347faa75e1747512c6b01 P a4ced4ae4cb143ec86ef7b0a99711d22 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } }
I20260808 18:22:35.459713 26254 leader_election.cc:288] T addbf42ee53347faa75e1747512c6b01 P a4ced4ae4cb143ec86ef7b0a99711d22 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8d738d4e749a48be988ae86c964d91c8 (127.23.72.129:33739), 6bd92cb899f44c2cbe8bc5be5655f22f (127.23.72.131:40149)
I20260808 18:22:35.463661 26246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:35.460124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:59617 (local address 127.23.72.131:40149)
0808 18:22:35.460294 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:35.460298 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:35.460402 (+ 104us) server_negotiation.cc:407] Connection header received
0808 18:22:35.460579 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:35.460582 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:35.460625 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:35.460740 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:22:35.461050 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.461563 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.462672 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.463076 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.463200 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:35.463257 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:35.463467 (+ 210us) server_negotiation.cc:299] Negotiation successful
0808 18:22:35.463542 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260808 18:22:35.463685 26258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:35.459944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.131:40149 (local address 127.23.72.130:59617)
0808 18:22:35.460309 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:35.460322 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:35.460433 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:35.460753 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:35.460757 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:35.460776 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:22:35.460917 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:35.460931 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.461678 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.461684 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:35.462538 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:35.462548 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.463200 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.463204 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:35.463310 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:35.463344 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 18:22:35.463433 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 18:22:35.463530 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"mutex_wait_us":18,"thread_start_us":128,"threads_started":1}
I20260808 18:22:35.463831 25958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:35.459895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:33739 (local address 127.23.72.130:55759)
0808 18:22:35.460098 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:35.460114 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:35.460211 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:35.460552 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:35.460554 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:35.460568 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:22:35.460679 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:35.460684 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.461493 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.461498 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:35.462696 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:35.462706 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.463351 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.463354 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:35.463516 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:35.463533 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 18:22:35.463619 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 18:22:35.463689 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":22}
I20260808 18:22:35.463815 26244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:35.459968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:55759 (local address 127.23.72.129:33739)
0808 18:22:35.460100 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:35.460105 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:35.460301 (+ 196us) server_negotiation.cc:407] Connection header received
0808 18:22:35.460366 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:35.460370 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:35.460426 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:35.460504 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:22:35.460825 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.461375 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.462832 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.463223 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.463351 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:35.463401 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:35.463637 (+ 236us) server_negotiation.cc:299] Negotiation successful
0808 18:22:35.463710 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 18:22:35.464322 26175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "addbf42ee53347faa75e1747512c6b01" candidate_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" is_pre_election: true
I20260808 18:22:35.464397 25864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "addbf42ee53347faa75e1747512c6b01" candidate_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d738d4e749a48be988ae86c964d91c8" is_pre_election: true
I20260808 18:22:35.464645 26175 raft_consensus.cc:2454] T addbf42ee53347faa75e1747512c6b01 P 6bd92cb899f44c2cbe8bc5be5655f22f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4ced4ae4cb143ec86ef7b0a99711d22 in term 0.
I20260808 18:22:35.464689 25864 raft_consensus.cc:2454] T addbf42ee53347faa75e1747512c6b01 P 8d738d4e749a48be988ae86c964d91c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4ced4ae4cb143ec86ef7b0a99711d22 in term 0.
I20260808 18:22:35.465147 25950 leader_election.cc:302] T addbf42ee53347faa75e1747512c6b01 P a4ced4ae4cb143ec86ef7b0a99711d22 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bd92cb899f44c2cbe8bc5be5655f22f, a4ced4ae4cb143ec86ef7b0a99711d22; no voters:
I20260808 18:22:35.465440 26254 raft_consensus.cc:2781] T addbf42ee53347faa75e1747512c6b01 P a4ced4ae4cb143ec86ef7b0a99711d22 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:22:35.465528 26254 raft_consensus.cc:482] T addbf42ee53347faa75e1747512c6b01 P a4ced4ae4cb143ec86ef7b0a99711d22 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:22:35.465570 26254 raft_consensus.cc:3037] T addbf42ee53347faa75e1747512c6b01 P a4ced4ae4cb143ec86ef7b0a99711d22 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:35.466838 26254 raft_consensus.cc:504] T addbf42ee53347faa75e1747512c6b01 P a4ced4ae4cb143ec86ef7b0a99711d22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } }
I20260808 18:22:35.467154 26254 leader_election.cc:288] T addbf42ee53347faa75e1747512c6b01 P a4ced4ae4cb143ec86ef7b0a99711d22 [CANDIDATE]: Term 1 election: Requested vote from peers 8d738d4e749a48be988ae86c964d91c8 (127.23.72.129:33739), 6bd92cb899f44c2cbe8bc5be5655f22f (127.23.72.131:40149)
I20260808 18:22:35.467532 25864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "addbf42ee53347faa75e1747512c6b01" candidate_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d738d4e749a48be988ae86c964d91c8"
I20260808 18:22:35.467561 26175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "addbf42ee53347faa75e1747512c6b01" candidate_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f"
I20260808 18:22:35.467657 26175 raft_consensus.cc:3037] T addbf42ee53347faa75e1747512c6b01 P 6bd92cb899f44c2cbe8bc5be5655f22f [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:35.467657 25864 raft_consensus.cc:3037] T addbf42ee53347faa75e1747512c6b01 P 8d738d4e749a48be988ae86c964d91c8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:35.468969 25864 raft_consensus.cc:2454] T addbf42ee53347faa75e1747512c6b01 P 8d738d4e749a48be988ae86c964d91c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4ced4ae4cb143ec86ef7b0a99711d22 in term 1.
I20260808 18:22:35.468969 26175 raft_consensus.cc:2454] T addbf42ee53347faa75e1747512c6b01 P 6bd92cb899f44c2cbe8bc5be5655f22f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4ced4ae4cb143ec86ef7b0a99711d22 in term 1.
I20260808 18:22:35.469264 25950 leader_election.cc:302] T addbf42ee53347faa75e1747512c6b01 P a4ced4ae4cb143ec86ef7b0a99711d22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bd92cb899f44c2cbe8bc5be5655f22f, a4ced4ae4cb143ec86ef7b0a99711d22; no voters:
I20260808 18:22:35.469468 26254 raft_consensus.cc:2781] T addbf42ee53347faa75e1747512c6b01 P a4ced4ae4cb143ec86ef7b0a99711d22 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:22:35.469725 26254 raft_consensus.cc:686] T addbf42ee53347faa75e1747512c6b01 P a4ced4ae4cb143ec86ef7b0a99711d22 [term 1 LEADER]: Becoming Leader. State: Replica: a4ced4ae4cb143ec86ef7b0a99711d22, State: Running, Role: LEADER
I20260808 18:22:35.470089 26254 consensus_queue.cc:237] T addbf42ee53347faa75e1747512c6b01 P a4ced4ae4cb143ec86ef7b0a99711d22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } }
I20260808 18:22:35.473132 25703 catalog_manager.cc:5495] T addbf42ee53347faa75e1747512c6b01 P a4ced4ae4cb143ec86ef7b0a99711d22 reported cstate change: term changed from 0 to 1, leader changed from <none> to a4ced4ae4cb143ec86ef7b0a99711d22 (127.23.72.130). New cstate: current_term: 1 leader_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } health_report { overall_health: UNKNOWN } } }
Aug 08 18:22:35 dist-test-slave-5k54 krb5kdc[25652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213354, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:22:35.517676 25755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:35.509266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39108 (local address 127.23.72.190:45825)
0808 18:22:35.509405 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:35.509412 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:35.509591 (+ 179us) server_negotiation.cc:407] Connection header received
0808 18:22:35.509750 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:35.509753 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:35.509796 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:35.509889 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:35.510574 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.511090 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.511906 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.512115 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.512220 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:35.514515 (+ 2295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:35.514519 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:35.514521 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:35.514541 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:35.515795 (+ 1254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:35.516327 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:35.516333 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:35.516335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:35.516410 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:35.516672 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:35.516676 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:35.516678 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:35.516894 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:35.517024 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:35.517361 (+ 337us) server_negotiation.cc:299] Negotiation successful
0808 18:22:35.517520 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":51}
I20260808 18:22:35.517678 26267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:35.509175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:45825 (local address 127.0.0.1:39108)
0808 18:22:35.509498 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:35.509510 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:35.509652 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:35.509906 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:35.509909 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:35.510342 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:35.510446 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:35.510453 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.511199 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.511205 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:35.511782 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:35.511791 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.512227 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.512231 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:35.512311 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:35.512862 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:35.512873 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:35.514372 (+ 1499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:35.515965 (+ 1593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:35.515976 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:35.515983 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:35.516206 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:35.516499 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:35.516504 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:35.516506 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:35.516585 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:35.517013 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:35.517019 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:35.517170 (+ 151us) client_negotiation.cc:769] Sending connection context
0808 18:22:35.517301 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 18:22:35.517465 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":92,"threads_started":1}
I20260808 18:22:35.521804 25703 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:39108:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 18:22:35.522593 25703 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:22:35.527674 26001 tablet_service.cc:1467] Processing CreateTablet for tablet b950a5263f244348b18309f9cbc8e04c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b25e33a7971149589b7c6eaf8cc286f3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 18:22:35.527696 25849 tablet_service.cc:1467] Processing CreateTablet for tablet b950a5263f244348b18309f9cbc8e04c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b25e33a7971149589b7c6eaf8cc286f3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 18:22:35.528475 26001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b950a5263f244348b18309f9cbc8e04c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:35.528475 25849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b950a5263f244348b18309f9cbc8e04c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:35.528235 26153 tablet_service.cc:1467] Processing CreateTablet for tablet b950a5263f244348b18309f9cbc8e04c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b25e33a7971149589b7c6eaf8cc286f3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 18:22:35.528823 26153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b950a5263f244348b18309f9cbc8e04c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:35.531046 26249 tablet_bootstrap.cc:492] T b950a5263f244348b18309f9cbc8e04c P a4ced4ae4cb143ec86ef7b0a99711d22: Bootstrap starting.
I20260808 18:22:35.531618 26250 tablet_bootstrap.cc:492] T b950a5263f244348b18309f9cbc8e04c P 8d738d4e749a48be988ae86c964d91c8: Bootstrap starting.
I20260808 18:22:35.532141 26249 tablet_bootstrap.cc:654] T b950a5263f244348b18309f9cbc8e04c P a4ced4ae4cb143ec86ef7b0a99711d22: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:35.532636 26250 tablet_bootstrap.cc:654] T b950a5263f244348b18309f9cbc8e04c P 8d738d4e749a48be988ae86c964d91c8: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:35.533296 26248 tablet_bootstrap.cc:492] T b950a5263f244348b18309f9cbc8e04c P 6bd92cb899f44c2cbe8bc5be5655f22f: Bootstrap starting.
I20260808 18:22:35.533511 26249 tablet_bootstrap.cc:492] T b950a5263f244348b18309f9cbc8e04c P a4ced4ae4cb143ec86ef7b0a99711d22: No bootstrap required, opened a new log
I20260808 18:22:35.533595 26249 ts_tablet_manager.cc:1397] T b950a5263f244348b18309f9cbc8e04c P a4ced4ae4cb143ec86ef7b0a99711d22: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 18:22:35.534031 26250 tablet_bootstrap.cc:492] T b950a5263f244348b18309f9cbc8e04c P 8d738d4e749a48be988ae86c964d91c8: No bootstrap required, opened a new log
I20260808 18:22:35.534107 26250 ts_tablet_manager.cc:1397] T b950a5263f244348b18309f9cbc8e04c P 8d738d4e749a48be988ae86c964d91c8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 18:22:35.534057 26249 raft_consensus.cc:348] T b950a5263f244348b18309f9cbc8e04c P a4ced4ae4cb143ec86ef7b0a99711d22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } }
I20260808 18:22:35.534178 26249 raft_consensus.cc:374] T b950a5263f244348b18309f9cbc8e04c P a4ced4ae4cb143ec86ef7b0a99711d22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:35.534221 26249 raft_consensus.cc:729] T b950a5263f244348b18309f9cbc8e04c P a4ced4ae4cb143ec86ef7b0a99711d22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4ced4ae4cb143ec86ef7b0a99711d22, State: Initialized, Role: FOLLOWER
I20260808 18:22:35.534267 26248 tablet_bootstrap.cc:654] T b950a5263f244348b18309f9cbc8e04c P 6bd92cb899f44c2cbe8bc5be5655f22f: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:35.534354 26249 consensus_queue.cc:260] T b950a5263f244348b18309f9cbc8e04c P a4ced4ae4cb143ec86ef7b0a99711d22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } }
I20260808 18:22:35.534588 26250 raft_consensus.cc:348] T b950a5263f244348b18309f9cbc8e04c P 8d738d4e749a48be988ae86c964d91c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } }
I20260808 18:22:35.534704 26250 raft_consensus.cc:374] T b950a5263f244348b18309f9cbc8e04c P 8d738d4e749a48be988ae86c964d91c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:35.534674 26249 ts_tablet_manager.cc:1428] T b950a5263f244348b18309f9cbc8e04c P a4ced4ae4cb143ec86ef7b0a99711d22: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 18:22:35.534739 26250 raft_consensus.cc:729] T b950a5263f244348b18309f9cbc8e04c P 8d738d4e749a48be988ae86c964d91c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d738d4e749a48be988ae86c964d91c8, State: Initialized, Role: FOLLOWER
I20260808 18:22:35.534659 26254 tablet_replica.cc:439] T b950a5263f244348b18309f9cbc8e04c P a4ced4ae4cb143ec86ef7b0a99711d22: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } } }
I20260808 18:22:35.534811 26254 tablet_replica.cc:442] T b950a5263f244348b18309f9cbc8e04c P a4ced4ae4cb143ec86ef7b0a99711d22: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:22:35.534876 26250 consensus_queue.cc:260] T b950a5263f244348b18309f9cbc8e04c P 8d738d4e749a48be988ae86c964d91c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } }
I20260808 18:22:35.535116 26250 ts_tablet_manager.cc:1428] T b950a5263f244348b18309f9cbc8e04c P 8d738d4e749a48be988ae86c964d91c8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 18:22:35.535255 26255 tablet_replica.cc:439] T b950a5263f244348b18309f9cbc8e04c P 8d738d4e749a48be988ae86c964d91c8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } } }
I20260808 18:22:35.535391 26255 tablet_replica.cc:442] T b950a5263f244348b18309f9cbc8e04c P 8d738d4e749a48be988ae86c964d91c8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:22:35.536366 26248 tablet_bootstrap.cc:492] T b950a5263f244348b18309f9cbc8e04c P 6bd92cb899f44c2cbe8bc5be5655f22f: No bootstrap required, opened a new log
I20260808 18:22:35.536473 26248 ts_tablet_manager.cc:1397] T b950a5263f244348b18309f9cbc8e04c P 6bd92cb899f44c2cbe8bc5be5655f22f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 18:22:35.536931 26248 raft_consensus.cc:348] T b950a5263f244348b18309f9cbc8e04c P 6bd92cb899f44c2cbe8bc5be5655f22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } }
I20260808 18:22:35.537047 26248 raft_consensus.cc:374] T b950a5263f244348b18309f9cbc8e04c P 6bd92cb899f44c2cbe8bc5be5655f22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:35.537108 26248 raft_consensus.cc:729] T b950a5263f244348b18309f9cbc8e04c P 6bd92cb899f44c2cbe8bc5be5655f22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bd92cb899f44c2cbe8bc5be5655f22f, State: Initialized, Role: FOLLOWER
I20260808 18:22:35.537235 26248 consensus_queue.cc:260] T b950a5263f244348b18309f9cbc8e04c P 6bd92cb899f44c2cbe8bc5be5655f22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } }
I20260808 18:22:35.537535 26248 ts_tablet_manager.cc:1428] T b950a5263f244348b18309f9cbc8e04c P 6bd92cb899f44c2cbe8bc5be5655f22f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 18:22:35.537528 26257 tablet_replica.cc:439] T b950a5263f244348b18309f9cbc8e04c P 6bd92cb899f44c2cbe8bc5be5655f22f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } } }
I20260808 18:22:35.537678 26257 tablet_replica.cc:442] T b950a5263f244348b18309f9cbc8e04c P 6bd92cb899f44c2cbe8bc5be5655f22f: This TxnStatusTablet replica's current role is: FOLLOWER
W20260808 18:22:35.569278 26225 tablet.cc:2369] T addbf42ee53347faa75e1747512c6b01 P 6bd92cb899f44c2cbe8bc5be5655f22f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 18:22:35.597487 26073 tablet.cc:2369] T addbf42ee53347faa75e1747512c6b01 P a4ced4ae4cb143ec86ef7b0a99711d22: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 18:22:35.633581 25924 tablet.cc:2369] T addbf42ee53347faa75e1747512c6b01 P 8d738d4e749a48be988ae86c964d91c8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:22:35.653687 26255 raft_consensus.cc:482] T b950a5263f244348b18309f9cbc8e04c P 8d738d4e749a48be988ae86c964d91c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:22:35.653877 26255 raft_consensus.cc:504] T b950a5263f244348b18309f9cbc8e04c P 8d738d4e749a48be988ae86c964d91c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } }
I20260808 18:22:35.655113 26255 leader_election.cc:288] T b950a5263f244348b18309f9cbc8e04c P 8d738d4e749a48be988ae86c964d91c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a4ced4ae4cb143ec86ef7b0a99711d22 (127.23.72.130:36341), 6bd92cb899f44c2cbe8bc5be5655f22f (127.23.72.131:40149)
I20260808 18:22:35.659222 26273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:35.655401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.130:36341 (local address 127.23.72.129:40205)
0808 18:22:35.655764 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:35.655782 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:35.655961 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:35.656389 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:35.656391 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:35.656408 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:22:35.656502 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:35.656506 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.657235 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.657240 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:35.658050 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:35.658059 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.658765 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.658769 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:35.658885 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:35.658904 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 18:22:35.658989 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 18:22:35.659067 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":150,"threads_started":1}
I20260808 18:22:35.659211 26247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:35.655479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:40205 (local address 127.23.72.130:36341)
0808 18:22:35.655648 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:35.655652 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:35.655906 (+ 254us) server_negotiation.cc:407] Connection header received
0808 18:22:35.656143 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:35.656146 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:35.656214 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:35.656257 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:22:35.656618 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.657121 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.658201 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.658626 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.658765 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:35.658818 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:35.659019 (+ 201us) server_negotiation.cc:299] Negotiation successful
0808 18:22:35.659091 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 18:22:35.659821 26274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:35.655877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.131:40149 (local address 127.23.72.129:49267)
0808 18:22:35.656237 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:35.656249 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:35.656388 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:35.656638 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:35.656641 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:35.656657 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:22:35.657207 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:35.657215 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.657937 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.657943 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:35.658650 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:35.658659 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.659310 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.659314 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:35.659439 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:35.659473 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 18:22:35.659590 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 18:22:35.659658 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":135,"threads_started":1}
I20260808 18:22:35.659852 26246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:35.656060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:49267 (local address 127.23.72.131:40149)
0808 18:22:35.656191 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:35.656195 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:35.656343 (+ 148us) server_negotiation.cc:407] Connection header received
0808 18:22:35.656480 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:35.656484 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:35.656528 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:35.656630 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:22:35.657333 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.657822 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.658772 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.659207 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.659319 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:35.659374 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:35.659646 (+ 272us) server_negotiation.cc:299] Negotiation successful
0808 18:22:35.659713 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 18:22:35.659838 26022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b950a5263f244348b18309f9cbc8e04c" candidate_uuid: "8d738d4e749a48be988ae86c964d91c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" is_pre_election: true
I20260808 18:22:35.660069 26022 raft_consensus.cc:2454] T b950a5263f244348b18309f9cbc8e04c P a4ced4ae4cb143ec86ef7b0a99711d22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d738d4e749a48be988ae86c964d91c8 in term 0.
I20260808 18:22:35.660331 26175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b950a5263f244348b18309f9cbc8e04c" candidate_uuid: "8d738d4e749a48be988ae86c964d91c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" is_pre_election: true
I20260808 18:22:35.660488 26175 raft_consensus.cc:2454] T b950a5263f244348b18309f9cbc8e04c P 6bd92cb899f44c2cbe8bc5be5655f22f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d738d4e749a48be988ae86c964d91c8 in term 0.
I20260808 18:22:35.660492 25798 leader_election.cc:302] T b950a5263f244348b18309f9cbc8e04c P 8d738d4e749a48be988ae86c964d91c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d738d4e749a48be988ae86c964d91c8, a4ced4ae4cb143ec86ef7b0a99711d22; no voters:
I20260808 18:22:35.660784 26255 raft_consensus.cc:2781] T b950a5263f244348b18309f9cbc8e04c P 8d738d4e749a48be988ae86c964d91c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:22:35.660864 26255 raft_consensus.cc:482] T b950a5263f244348b18309f9cbc8e04c P 8d738d4e749a48be988ae86c964d91c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:22:35.660905 26255 raft_consensus.cc:3037] T b950a5263f244348b18309f9cbc8e04c P 8d738d4e749a48be988ae86c964d91c8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:35.661911 26255 raft_consensus.cc:504] T b950a5263f244348b18309f9cbc8e04c P 8d738d4e749a48be988ae86c964d91c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } }
I20260808 18:22:35.662223 26255 leader_election.cc:288] T b950a5263f244348b18309f9cbc8e04c P 8d738d4e749a48be988ae86c964d91c8 [CANDIDATE]: Term 1 election: Requested vote from peers a4ced4ae4cb143ec86ef7b0a99711d22 (127.23.72.130:36341), 6bd92cb899f44c2cbe8bc5be5655f22f (127.23.72.131:40149)
I20260808 18:22:35.662550 26022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b950a5263f244348b18309f9cbc8e04c" candidate_uuid: "8d738d4e749a48be988ae86c964d91c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22"
I20260808 18:22:35.662616 26175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b950a5263f244348b18309f9cbc8e04c" candidate_uuid: "8d738d4e749a48be988ae86c964d91c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f"
I20260808 18:22:35.662689 26022 raft_consensus.cc:3037] T b950a5263f244348b18309f9cbc8e04c P a4ced4ae4cb143ec86ef7b0a99711d22 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:35.662742 26175 raft_consensus.cc:3037] T b950a5263f244348b18309f9cbc8e04c P 6bd92cb899f44c2cbe8bc5be5655f22f [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:35.663717 26022 raft_consensus.cc:2454] T b950a5263f244348b18309f9cbc8e04c P a4ced4ae4cb143ec86ef7b0a99711d22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d738d4e749a48be988ae86c964d91c8 in term 1.
I20260808 18:22:35.663738 26175 raft_consensus.cc:2454] T b950a5263f244348b18309f9cbc8e04c P 6bd92cb899f44c2cbe8bc5be5655f22f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d738d4e749a48be988ae86c964d91c8 in term 1.
I20260808 18:22:35.663983 25798 leader_election.cc:302] T b950a5263f244348b18309f9cbc8e04c P 8d738d4e749a48be988ae86c964d91c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d738d4e749a48be988ae86c964d91c8, a4ced4ae4cb143ec86ef7b0a99711d22; no voters:
I20260808 18:22:35.664165 26255 raft_consensus.cc:2781] T b950a5263f244348b18309f9cbc8e04c P 8d738d4e749a48be988ae86c964d91c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:22:35.664408 26255 raft_consensus.cc:686] T b950a5263f244348b18309f9cbc8e04c P 8d738d4e749a48be988ae86c964d91c8 [term 1 LEADER]: Becoming Leader. State: Replica: 8d738d4e749a48be988ae86c964d91c8, State: Running, Role: LEADER
I20260808 18:22:35.664726 26255 consensus_queue.cc:237] T b950a5263f244348b18309f9cbc8e04c P 8d738d4e749a48be988ae86c964d91c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } }
I20260808 18:22:35.667061 26275 tablet_replica.cc:439] T b950a5263f244348b18309f9cbc8e04c P 8d738d4e749a48be988ae86c964d91c8: TxnStatusTablet state changed. Reason: New leader 8d738d4e749a48be988ae86c964d91c8. Latest consensus state: current_term: 1 leader_uuid: "8d738d4e749a48be988ae86c964d91c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } } }
I20260808 18:22:35.667171 26275 tablet_replica.cc:442] T b950a5263f244348b18309f9cbc8e04c P 8d738d4e749a48be988ae86c964d91c8: This TxnStatusTablet replica's current role is: LEADER
I20260808 18:22:35.667498 26277 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 18:22:35.667706 26277 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 18:22:35.668093 25704 catalog_manager.cc:5495] T b950a5263f244348b18309f9cbc8e04c P 8d738d4e749a48be988ae86c964d91c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8d738d4e749a48be988ae86c964d91c8 (127.23.72.129). New cstate: current_term: 1 leader_uuid: "8d738d4e749a48be988ae86c964d91c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } health_report { overall_health: UNKNOWN } } }
I20260808 18:22:35.690138 26267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:35.685885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:33739 (local address 127.0.0.1:46064)
0808 18:22:35.686050 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:35.686069 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:35.686176 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:35.686520 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:35.686523 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:35.686531 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 18:22:35.686638 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:35.686644 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.687725 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.687728 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:35.688414 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:35.688424 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.688982 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.688985 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:35.689055 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:35.689098 (+ 43us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 18:22:35.689864 (+ 766us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 18:22:35.689875 (+ 11us) client_negotiation.cc:769] Sending connection context
0808 18:22:35.689926 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 18:22:35.690009 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 18:22:35.690158 26244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:35.686033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46064 (local address 127.23.72.129:33739)
0808 18:22:35.686154 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:35.686158 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:35.686172 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:22:35.686338 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:35.686341 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:35.686401 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:35.686503 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:22:35.686785 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.687615 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.688526 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.688870 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.688975 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:35.689233 (+ 258us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:22:35.689766 (+ 533us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:22:35.689878 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:35.689939 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 18:22:35.690010 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 18:22:35.695156 26022 raft_consensus.cc:1261] T b950a5263f244348b18309f9cbc8e04c P a4ced4ae4cb143ec86ef7b0a99711d22 [term 1 FOLLOWER]: Refusing update from remote peer 8d738d4e749a48be988ae86c964d91c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:22:35.695191 26175 raft_consensus.cc:1261] T b950a5263f244348b18309f9cbc8e04c P 6bd92cb899f44c2cbe8bc5be5655f22f [term 1 FOLLOWER]: Refusing update from remote peer 8d738d4e749a48be988ae86c964d91c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:22:35.695852 26255 consensus_queue.cc:1035] T b950a5263f244348b18309f9cbc8e04c P 8d738d4e749a48be988ae86c964d91c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:22:35.696046 26275 consensus_queue.cc:1035] T b950a5263f244348b18309f9cbc8e04c P 8d738d4e749a48be988ae86c964d91c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:22:35.698887 26254 tablet_replica.cc:439] T b950a5263f244348b18309f9cbc8e04c P a4ced4ae4cb143ec86ef7b0a99711d22: TxnStatusTablet state changed. Reason: New leader 8d738d4e749a48be988ae86c964d91c8. Latest consensus state: current_term: 1 leader_uuid: "8d738d4e749a48be988ae86c964d91c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } } }
I20260808 18:22:35.698997 26254 tablet_replica.cc:442] T b950a5263f244348b18309f9cbc8e04c P a4ced4ae4cb143ec86ef7b0a99711d22: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:22:35.700117 26257 tablet_replica.cc:439] T b950a5263f244348b18309f9cbc8e04c P 6bd92cb899f44c2cbe8bc5be5655f22f: TxnStatusTablet state changed. Reason: New leader 8d738d4e749a48be988ae86c964d91c8. Latest consensus state: current_term: 1 leader_uuid: "8d738d4e749a48be988ae86c964d91c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } } }
I20260808 18:22:35.700237 26257 tablet_replica.cc:442] T b950a5263f244348b18309f9cbc8e04c P 6bd92cb899f44c2cbe8bc5be5655f22f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:22:35.702926 26255 tablet_replica.cc:439] T b950a5263f244348b18309f9cbc8e04c P 8d738d4e749a48be988ae86c964d91c8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8d738d4e749a48be988ae86c964d91c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } } }
I20260808 18:22:35.703040 26255 tablet_replica.cc:442] T b950a5263f244348b18309f9cbc8e04c P 8d738d4e749a48be988ae86c964d91c8: This TxnStatusTablet replica's current role is: LEADER
I20260808 18:22:35.703199 26283 mvcc.cc:205] Tried to move back new op lower bound from 7316329904920420352 to 7316329904805060608. Current Snapshot: MvccSnapshot[applied={T|T < 7316329904920420352}]
I20260808 18:22:35.703652 26254 tablet_replica.cc:439] T b950a5263f244348b18309f9cbc8e04c P a4ced4ae4cb143ec86ef7b0a99711d22: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8d738d4e749a48be988ae86c964d91c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } } }
I20260808 18:22:35.703756 26254 tablet_replica.cc:442] T b950a5263f244348b18309f9cbc8e04c P a4ced4ae4cb143ec86ef7b0a99711d22: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:22:35.703873 26281 mvcc.cc:205] Tried to move back new op lower bound from 7316329904920420352 to 7316329904805060608. Current Snapshot: MvccSnapshot[applied={T|T < 7316329904920420352}]
I20260808 18:22:35.706578 26275 tablet_replica.cc:439] T b950a5263f244348b18309f9cbc8e04c P 8d738d4e749a48be988ae86c964d91c8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8d738d4e749a48be988ae86c964d91c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } } }
I20260808 18:22:35.706704 26275 tablet_replica.cc:442] T b950a5263f244348b18309f9cbc8e04c P 8d738d4e749a48be988ae86c964d91c8: This TxnStatusTablet replica's current role is: LEADER
I20260808 18:22:35.708801 26282 mvcc.cc:205] Tried to move back new op lower bound from 7316329904920420352 to 7316329904805060608. Current Snapshot: MvccSnapshot[applied={T|T < 7316329904920420352}]
I20260808 18:22:35.709615 26257 tablet_replica.cc:439] T b950a5263f244348b18309f9cbc8e04c P 6bd92cb899f44c2cbe8bc5be5655f22f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8d738d4e749a48be988ae86c964d91c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } } }
I20260808 18:22:35.709726 26257 tablet_replica.cc:442] T b950a5263f244348b18309f9cbc8e04c P 6bd92cb899f44c2cbe8bc5be5655f22f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:22:35.715361 26247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:35.711713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52574 (local address 127.23.72.130:36341)
0808 18:22:35.711820 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:35.711825 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:35.711933 (+ 108us) server_negotiation.cc:407] Connection header received
0808 18:22:35.712129 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:35.712131 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:35.712195 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:35.712302 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:22:35.712569 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.713079 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.713948 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.714187 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.714305 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:35.714583 (+ 278us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:22:35.714972 (+ 389us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:22:35.715095 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:35.715161 (+ 66us) server_negotiation.cc:299] Negotiation successful
0808 18:22:35.715213 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 18:22:35.726886 26244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:35.723187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:52921 (local address 127.23.72.129:33739)
0808 18:22:35.723351 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:35.723356 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:35.723415 (+ 59us) server_negotiation.cc:407] Connection header received
0808 18:22:35.723587 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:35.723590 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:35.723642 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:35.723737 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:22:35.724104 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.724602 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.725816 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.726038 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.726128 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:35.726486 (+ 358us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:22:35.726554 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:22:35.726612 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:35.726730 (+ 118us) server_negotiation.cc:299] Negotiation successful
0808 18:22:35.726785 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 18:22:35.726941 26293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:35.723042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:33739 (local address 127.23.72.130:52921)
0808 18:22:35.723327 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:35.723341 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:35.723474 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:35.723847 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:35.723851 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:35.723859 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 18:22:35.723977 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:35.723983 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.724702 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.724705 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:35.725657 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:35.725670 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.726167 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.726171 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:35.726265 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:35.726320 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 18:22:35.726637 (+ 317us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 18:22:35.726643 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 18:22:35.726707 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 18:22:35.726786 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":88,"threads_started":1}
I20260808 18:22:35.734542 25864 raft_consensus.cc:1261] T addbf42ee53347faa75e1747512c6b01 P 8d738d4e749a48be988ae86c964d91c8 [term 1 FOLLOWER]: Refusing update from remote peer a4ced4ae4cb143ec86ef7b0a99711d22: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:22:35.735003 26175 raft_consensus.cc:1261] T addbf42ee53347faa75e1747512c6b01 P 6bd92cb899f44c2cbe8bc5be5655f22f [term 1 FOLLOWER]: Refusing update from remote peer a4ced4ae4cb143ec86ef7b0a99711d22: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:22:35.735179 26254 consensus_queue.cc:1035] T addbf42ee53347faa75e1747512c6b01 P a4ced4ae4cb143ec86ef7b0a99711d22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:22:35.735489 26259 consensus_queue.cc:1035] T addbf42ee53347faa75e1747512c6b01 P a4ced4ae4cb143ec86ef7b0a99711d22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:22:35.757886 26247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:35.753960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:57447 (local address 127.23.72.130:36341)
0808 18:22:35.754096 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:35.754101 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:35.754343 (+ 242us) server_negotiation.cc:407] Connection header received
0808 18:22:35.754432 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:35.754434 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:35.754475 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:35.754540 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:22:35.754854 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.755556 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.756833 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.757041 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.757126 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:35.757468 (+ 342us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:22:35.757531 (+ 63us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:22:35.757579 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:35.757698 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 18:22:35.757740 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 18:22:35.757920 26298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:35.753901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.130:36341 (local address 127.23.72.129:57447)
0808 18:22:35.754221 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:35.754237 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:35.754353 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:35.754578 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:35.754582 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:35.754590 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 18:22:35.754719 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:35.754726 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.755657 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.755660 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:35.756723 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:35.756735 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.757182 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.757186 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:35.757287 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:35.757339 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 18:22:35.757608 (+ 269us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 18:22:35.757614 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 18:22:35.757682 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 18:22:35.757758 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":118,"threads_started":1}
I20260808 18:22:35.787101 25704 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:22:35.787305 25704 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:22:35.790066 25704 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9d816eb59738467ebf0d0d6cbf7bbcaf: Sending DeleteTablet for 3 replicas of tablet addbf42ee53347faa75e1747512c6b01
I20260808 18:22:35.790843 25849 tablet_service.cc:1514] Processing DeleteTablet for tablet addbf42ee53347faa75e1747512c6b01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:22:35 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:46058
I20260808 18:22:35.791106 26153 tablet_service.cc:1514] Processing DeleteTablet for tablet addbf42ee53347faa75e1747512c6b01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:22:35 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:35138
I20260808 18:22:35.792145 26001 tablet_service.cc:1514] Processing DeleteTablet for tablet addbf42ee53347faa75e1747512c6b01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:22:35 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:52570
I20260808 18:22:35.792614 26305 tablet_replica.cc:330] T addbf42ee53347faa75e1747512c6b01 P a4ced4ae4cb143ec86ef7b0a99711d22: stopping tablet replica
I20260808 18:22:35.792642 25704 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39092:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:22:35.793051 25704 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:22:35.793211 26305 raft_consensus.cc:2229] T addbf42ee53347faa75e1747512c6b01 P a4ced4ae4cb143ec86ef7b0a99711d22 [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:22:35.793641 26305 raft_consensus.cc:2258] T addbf42ee53347faa75e1747512c6b01 P a4ced4ae4cb143ec86ef7b0a99711d22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:22:35.795107 26305 ts_tablet_manager.cc:1905] T addbf42ee53347faa75e1747512c6b01 P a4ced4ae4cb143ec86ef7b0a99711d22: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:22:35.798454 25849 tablet_service.cc:1467] Processing CreateTablet for tablet 36014ba00a0847dd89300a82e757d7d8 (DEFAULT_TABLE table=test-table [id=1928a4e77cb8411486699a4be2b3f1d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:22:35.798508 26305 ts_tablet_manager.cc:1918] T addbf42ee53347faa75e1747512c6b01 P a4ced4ae4cb143ec86ef7b0a99711d22: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 18:22:35.798730 25849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36014ba00a0847dd89300a82e757d7d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:35.798738 26305 log.cc:1196] T addbf42ee53347faa75e1747512c6b01 P a4ced4ae4cb143ec86ef7b0a99711d22: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-1/wal/wals/addbf42ee53347faa75e1747512c6b01
I20260808 18:22:35.799183 26305 ts_tablet_manager.cc:1939] T addbf42ee53347faa75e1747512c6b01 P a4ced4ae4cb143ec86ef7b0a99711d22: Deleting consensus metadata
I20260808 18:22:35.800423 25687 catalog_manager.cc:4841] TS a4ced4ae4cb143ec86ef7b0a99711d22 (127.23.72.130:36341): tablet addbf42ee53347faa75e1747512c6b01 (table test-table [id=1083a8e507d54a6c9da884edb511b98c]) successfully deleted
I20260808 18:22:35.801152 26250 tablet_bootstrap.cc:492] T 36014ba00a0847dd89300a82e757d7d8 P 8d738d4e749a48be988ae86c964d91c8: Bootstrap starting.
I20260808 18:22:35.801290 26001 tablet_service.cc:1467] Processing CreateTablet for tablet 36014ba00a0847dd89300a82e757d7d8 (DEFAULT_TABLE table=test-table [id=1928a4e77cb8411486699a4be2b3f1d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:22:35.801519 26001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36014ba00a0847dd89300a82e757d7d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:35.802178 26250 tablet_bootstrap.cc:654] T 36014ba00a0847dd89300a82e757d7d8 P 8d738d4e749a48be988ae86c964d91c8: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:35.803663 26153 tablet_service.cc:1467] Processing CreateTablet for tablet 36014ba00a0847dd89300a82e757d7d8 (DEFAULT_TABLE table=test-table [id=1928a4e77cb8411486699a4be2b3f1d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:22:35.803925 26153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36014ba00a0847dd89300a82e757d7d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:35.806226 26248 tablet_bootstrap.cc:492] T 36014ba00a0847dd89300a82e757d7d8 P 6bd92cb899f44c2cbe8bc5be5655f22f: Bootstrap starting.
I20260808 18:22:35.807147 26248 tablet_bootstrap.cc:654] T 36014ba00a0847dd89300a82e757d7d8 P 6bd92cb899f44c2cbe8bc5be5655f22f: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:35.807250 26304 tablet_replica.cc:330] T addbf42ee53347faa75e1747512c6b01 P 6bd92cb899f44c2cbe8bc5be5655f22f: stopping tablet replica
I20260808 18:22:35.807560 26249 tablet_bootstrap.cc:492] T 36014ba00a0847dd89300a82e757d7d8 P a4ced4ae4cb143ec86ef7b0a99711d22: Bootstrap starting.
I20260808 18:22:35.807583 26304 raft_consensus.cc:2229] T addbf42ee53347faa75e1747512c6b01 P 6bd92cb899f44c2cbe8bc5be5655f22f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:22:35.807780 26304 raft_consensus.cc:2258] T addbf42ee53347faa75e1747512c6b01 P 6bd92cb899f44c2cbe8bc5be5655f22f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:22:35.807960 26303 tablet_replica.cc:330] T addbf42ee53347faa75e1747512c6b01 P 8d738d4e749a48be988ae86c964d91c8: stopping tablet replica
I20260808 18:22:35.808215 26303 raft_consensus.cc:2229] T addbf42ee53347faa75e1747512c6b01 P 8d738d4e749a48be988ae86c964d91c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:22:35.808377 26303 raft_consensus.cc:2258] T addbf42ee53347faa75e1747512c6b01 P 8d738d4e749a48be988ae86c964d91c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:22:35.808475 26249 tablet_bootstrap.cc:654] T 36014ba00a0847dd89300a82e757d7d8 P a4ced4ae4cb143ec86ef7b0a99711d22: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:35.810271 26250 tablet_bootstrap.cc:492] T 36014ba00a0847dd89300a82e757d7d8 P 8d738d4e749a48be988ae86c964d91c8: No bootstrap required, opened a new log
I20260808 18:22:35.810395 26250 ts_tablet_manager.cc:1397] T 36014ba00a0847dd89300a82e757d7d8 P 8d738d4e749a48be988ae86c964d91c8: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.000s
I20260808 18:22:35.810478 26303 ts_tablet_manager.cc:1905] T addbf42ee53347faa75e1747512c6b01 P 8d738d4e749a48be988ae86c964d91c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:22:35.810833 26250 raft_consensus.cc:348] T 36014ba00a0847dd89300a82e757d7d8 P 8d738d4e749a48be988ae86c964d91c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } }
I20260808 18:22:35.810950 26250 raft_consensus.cc:374] T 36014ba00a0847dd89300a82e757d7d8 P 8d738d4e749a48be988ae86c964d91c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:35.810986 26250 raft_consensus.cc:729] T 36014ba00a0847dd89300a82e757d7d8 P 8d738d4e749a48be988ae86c964d91c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d738d4e749a48be988ae86c964d91c8, State: Initialized, Role: FOLLOWER
I20260808 18:22:35.811286 26250 consensus_queue.cc:260] T 36014ba00a0847dd89300a82e757d7d8 P 8d738d4e749a48be988ae86c964d91c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } }
I20260808 18:22:35.813282 26304 ts_tablet_manager.cc:1905] T addbf42ee53347faa75e1747512c6b01 P 6bd92cb899f44c2cbe8bc5be5655f22f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:22:35.813459 26250 ts_tablet_manager.cc:1428] T 36014ba00a0847dd89300a82e757d7d8 P 8d738d4e749a48be988ae86c964d91c8: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260808 18:22:35.815824 26249 tablet_bootstrap.cc:492] T 36014ba00a0847dd89300a82e757d7d8 P a4ced4ae4cb143ec86ef7b0a99711d22: No bootstrap required, opened a new log
I20260808 18:22:35.815915 26249 ts_tablet_manager.cc:1397] T 36014ba00a0847dd89300a82e757d7d8 P a4ced4ae4cb143ec86ef7b0a99711d22: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.001s
I20260808 18:22:35.816402 26249 raft_consensus.cc:348] T 36014ba00a0847dd89300a82e757d7d8 P a4ced4ae4cb143ec86ef7b0a99711d22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } }
I20260808 18:22:35.816529 26249 raft_consensus.cc:374] T 36014ba00a0847dd89300a82e757d7d8 P a4ced4ae4cb143ec86ef7b0a99711d22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:35.816565 26249 raft_consensus.cc:729] T 36014ba00a0847dd89300a82e757d7d8 P a4ced4ae4cb143ec86ef7b0a99711d22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4ced4ae4cb143ec86ef7b0a99711d22, State: Initialized, Role: FOLLOWER
I20260808 18:22:35.816682 26249 consensus_queue.cc:260] T 36014ba00a0847dd89300a82e757d7d8 P a4ced4ae4cb143ec86ef7b0a99711d22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } }
I20260808 18:22:35.816967 26249 ts_tablet_manager.cc:1428] T 36014ba00a0847dd89300a82e757d7d8 P a4ced4ae4cb143ec86ef7b0a99711d22: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 18:22:35.819342 26248 tablet_bootstrap.cc:492] T 36014ba00a0847dd89300a82e757d7d8 P 6bd92cb899f44c2cbe8bc5be5655f22f: No bootstrap required, opened a new log
I20260808 18:22:35.819471 26248 ts_tablet_manager.cc:1397] T 36014ba00a0847dd89300a82e757d7d8 P 6bd92cb899f44c2cbe8bc5be5655f22f: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20260808 18:22:35.819751 26304 ts_tablet_manager.cc:1918] T addbf42ee53347faa75e1747512c6b01 P 6bd92cb899f44c2cbe8bc5be5655f22f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 18:22:35.819837 26304 log.cc:1196] T addbf42ee53347faa75e1747512c6b01 P 6bd92cb899f44c2cbe8bc5be5655f22f: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-2/wal/wals/addbf42ee53347faa75e1747512c6b01
I20260808 18:22:35.819949 26248 raft_consensus.cc:348] T 36014ba00a0847dd89300a82e757d7d8 P 6bd92cb899f44c2cbe8bc5be5655f22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } }
I20260808 18:22:35.820078 26248 raft_consensus.cc:374] T 36014ba00a0847dd89300a82e757d7d8 P 6bd92cb899f44c2cbe8bc5be5655f22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:35.820114 26248 raft_consensus.cc:729] T 36014ba00a0847dd89300a82e757d7d8 P 6bd92cb899f44c2cbe8bc5be5655f22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bd92cb899f44c2cbe8bc5be5655f22f, State: Initialized, Role: FOLLOWER
I20260808 18:22:35.820256 26248 consensus_queue.cc:260] T 36014ba00a0847dd89300a82e757d7d8 P 6bd92cb899f44c2cbe8bc5be5655f22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } }
I20260808 18:22:35.820572 26248 ts_tablet_manager.cc:1428] T 36014ba00a0847dd89300a82e757d7d8 P 6bd92cb899f44c2cbe8bc5be5655f22f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:22:35.820379 26303 ts_tablet_manager.cc:1918] T addbf42ee53347faa75e1747512c6b01 P 8d738d4e749a48be988ae86c964d91c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 18:22:35.822359 26303 log.cc:1196] T addbf42ee53347faa75e1747512c6b01 P 8d738d4e749a48be988ae86c964d91c8: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-0/wal/wals/addbf42ee53347faa75e1747512c6b01
I20260808 18:22:35.822728 26303 ts_tablet_manager.cc:1939] T addbf42ee53347faa75e1747512c6b01 P 8d738d4e749a48be988ae86c964d91c8: Deleting consensus metadata
I20260808 18:22:35.824292 26304 ts_tablet_manager.cc:1939] T addbf42ee53347faa75e1747512c6b01 P 6bd92cb899f44c2cbe8bc5be5655f22f: Deleting consensus metadata
I20260808 18:22:35.825613 25687 catalog_manager.cc:4841] TS 6bd92cb899f44c2cbe8bc5be5655f22f (127.23.72.131:40149): tablet addbf42ee53347faa75e1747512c6b01 (table test-table [id=1083a8e507d54a6c9da884edb511b98c]) successfully deleted
I20260808 18:22:35.826295 25686 catalog_manager.cc:4841] TS 8d738d4e749a48be988ae86c964d91c8 (127.23.72.129:33739): tablet addbf42ee53347faa75e1747512c6b01 (table test-table [id=1083a8e507d54a6c9da884edb511b98c]) successfully deleted
I20260808 18:22:35.900610 26288 raft_consensus.cc:482] T 36014ba00a0847dd89300a82e757d7d8 P 8d738d4e749a48be988ae86c964d91c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:22:35.900758 26288 raft_consensus.cc:504] T 36014ba00a0847dd89300a82e757d7d8 P 8d738d4e749a48be988ae86c964d91c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } }
I20260808 18:22:35.901158 26288 leader_election.cc:288] T 36014ba00a0847dd89300a82e757d7d8 P 8d738d4e749a48be988ae86c964d91c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a4ced4ae4cb143ec86ef7b0a99711d22 (127.23.72.130:36341), 6bd92cb899f44c2cbe8bc5be5655f22f (127.23.72.131:40149)
I20260808 18:22:35.901719 26175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36014ba00a0847dd89300a82e757d7d8" candidate_uuid: "8d738d4e749a48be988ae86c964d91c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" is_pre_election: true
I20260808 18:22:35.901723 26022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36014ba00a0847dd89300a82e757d7d8" candidate_uuid: "8d738d4e749a48be988ae86c964d91c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" is_pre_election: true
I20260808 18:22:35.901899 26175 raft_consensus.cc:2454] T 36014ba00a0847dd89300a82e757d7d8 P 6bd92cb899f44c2cbe8bc5be5655f22f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d738d4e749a48be988ae86c964d91c8 in term 0.
I20260808 18:22:35.901894 26022 raft_consensus.cc:2454] T 36014ba00a0847dd89300a82e757d7d8 P a4ced4ae4cb143ec86ef7b0a99711d22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d738d4e749a48be988ae86c964d91c8 in term 0.
I20260808 18:22:35.902236 25798 leader_election.cc:302] T 36014ba00a0847dd89300a82e757d7d8 P 8d738d4e749a48be988ae86c964d91c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d738d4e749a48be988ae86c964d91c8, a4ced4ae4cb143ec86ef7b0a99711d22; no voters:
I20260808 18:22:35.902415 26288 raft_consensus.cc:2781] T 36014ba00a0847dd89300a82e757d7d8 P 8d738d4e749a48be988ae86c964d91c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:22:35.902477 26288 raft_consensus.cc:482] T 36014ba00a0847dd89300a82e757d7d8 P 8d738d4e749a48be988ae86c964d91c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:22:35.902516 26288 raft_consensus.cc:3037] T 36014ba00a0847dd89300a82e757d7d8 P 8d738d4e749a48be988ae86c964d91c8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:35.903890 26288 raft_consensus.cc:504] T 36014ba00a0847dd89300a82e757d7d8 P 8d738d4e749a48be988ae86c964d91c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } }
I20260808 18:22:35.904238 26288 leader_election.cc:288] T 36014ba00a0847dd89300a82e757d7d8 P 8d738d4e749a48be988ae86c964d91c8 [CANDIDATE]: Term 1 election: Requested vote from peers a4ced4ae4cb143ec86ef7b0a99711d22 (127.23.72.130:36341), 6bd92cb899f44c2cbe8bc5be5655f22f (127.23.72.131:40149)
I20260808 18:22:35.904573 26022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36014ba00a0847dd89300a82e757d7d8" candidate_uuid: "8d738d4e749a48be988ae86c964d91c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22"
I20260808 18:22:35.904640 26175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36014ba00a0847dd89300a82e757d7d8" candidate_uuid: "8d738d4e749a48be988ae86c964d91c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f"
I20260808 18:22:35.904706 26022 raft_consensus.cc:3037] T 36014ba00a0847dd89300a82e757d7d8 P a4ced4ae4cb143ec86ef7b0a99711d22 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:35.904796 26175 raft_consensus.cc:3037] T 36014ba00a0847dd89300a82e757d7d8 P 6bd92cb899f44c2cbe8bc5be5655f22f [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:35.905673 26175 raft_consensus.cc:2454] T 36014ba00a0847dd89300a82e757d7d8 P 6bd92cb899f44c2cbe8bc5be5655f22f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d738d4e749a48be988ae86c964d91c8 in term 1.
I20260808 18:22:35.905921 26022 raft_consensus.cc:2454] T 36014ba00a0847dd89300a82e757d7d8 P a4ced4ae4cb143ec86ef7b0a99711d22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d738d4e749a48be988ae86c964d91c8 in term 1.
I20260808 18:22:35.905951 25798 leader_election.cc:302] T 36014ba00a0847dd89300a82e757d7d8 P 8d738d4e749a48be988ae86c964d91c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bd92cb899f44c2cbe8bc5be5655f22f, 8d738d4e749a48be988ae86c964d91c8; no voters:
I20260808 18:22:35.906134 26288 raft_consensus.cc:2781] T 36014ba00a0847dd89300a82e757d7d8 P 8d738d4e749a48be988ae86c964d91c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:22:35.906244 26288 raft_consensus.cc:686] T 36014ba00a0847dd89300a82e757d7d8 P 8d738d4e749a48be988ae86c964d91c8 [term 1 LEADER]: Becoming Leader. State: Replica: 8d738d4e749a48be988ae86c964d91c8, State: Running, Role: LEADER
I20260808 18:22:35.906390 26288 consensus_queue.cc:237] T 36014ba00a0847dd89300a82e757d7d8 P 8d738d4e749a48be988ae86c964d91c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } }
I20260808 18:22:35.908125 25704 catalog_manager.cc:5495] T 36014ba00a0847dd89300a82e757d7d8 P 8d738d4e749a48be988ae86c964d91c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8d738d4e749a48be988ae86c964d91c8 (127.23.72.129). New cstate: current_term: 1 leader_uuid: "8d738d4e749a48be988ae86c964d91c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } health_report { overall_health: UNKNOWN } } }
I20260808 18:22:35.940829 26244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:35.937467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46070 (local address 127.23.72.129:33739)
0808 18:22:35.937572 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:35.937577 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:35.937623 (+ 46us) server_negotiation.cc:407] Connection header received
0808 18:22:35.937788 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:35.937791 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:35.937843 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:35.937972 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:22:35.938224 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.938862 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.939778 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.940015 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.940126 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:35.940376 (+ 250us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:22:35.940457 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:22:35.940533 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:35.940624 (+ 91us) server_negotiation.cc:299] Negotiation successful
0808 18:22:35.940682 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 18:22:35.950030 26244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:35.946847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:45933 (local address 127.23.72.129:33739)
0808 18:22:35.946978 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:35.946981 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:35.946993 (+ 12us) server_negotiation.cc:407] Connection header received
0808 18:22:35.947098 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:35.947100 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:35.947137 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:35.947197 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:22:35.947470 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.947987 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.949148 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.949358 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.949429 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:35.949663 (+ 234us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:22:35.949721 (+ 58us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:22:35.949770 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:35.949873 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 18:22:35.949910 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 18:22:35.950141 26298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:35.946774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:33739 (local address 127.23.72.129:45933)
0808 18:22:35.946930 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:35.946944 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:35.947013 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:35.947232 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:35.947235 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:35.947242 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 18:22:35.947339 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:35.947345 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.948097 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.948101 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:35.949037 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:35.949046 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:35.949453 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:35.949456 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:35.949537 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:35.949566 (+ 29us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 18:22:35.949791 (+ 225us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 18:22:35.949796 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 18:22:35.949861 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 18:22:35.949936 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260808 18:22:35.955942 26175 raft_consensus.cc:1261] T 36014ba00a0847dd89300a82e757d7d8 P 6bd92cb899f44c2cbe8bc5be5655f22f [term 1 FOLLOWER]: Refusing update from remote peer 8d738d4e749a48be988ae86c964d91c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:22:35.955991 26022 raft_consensus.cc:1261] T 36014ba00a0847dd89300a82e757d7d8 P a4ced4ae4cb143ec86ef7b0a99711d22 [term 1 FOLLOWER]: Refusing update from remote peer 8d738d4e749a48be988ae86c964d91c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:22:35.956454 26255 consensus_queue.cc:1035] T 36014ba00a0847dd89300a82e757d7d8 P 8d738d4e749a48be988ae86c964d91c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:22:35.956637 26288 consensus_queue.cc:1035] T 36014ba00a0847dd89300a82e757d7d8 P 8d738d4e749a48be988ae86c964d91c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 18:22:35.969277 26296 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 18:22:37.523689 25704 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:22:37.523845 25704 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:22:37.526201 25704 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9d816eb59738467ebf0d0d6cbf7bbcaf: Sending DeleteTablet for 3 replicas of tablet 36014ba00a0847dd89300a82e757d7d8
I20260808 18:22:37.526716 25852 tablet_service.cc:1514] Processing DeleteTablet for tablet 36014ba00a0847dd89300a82e757d7d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:22:37 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:46058
I20260808 18:22:37.526784 26001 tablet_service.cc:1514] Processing DeleteTablet for tablet 36014ba00a0847dd89300a82e757d7d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:22:37 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:52570
I20260808 18:22:37.526925 26153 tablet_service.cc:1514] Processing DeleteTablet for tablet 36014ba00a0847dd89300a82e757d7d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:22:37 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:35138
I20260808 18:22:37.527092 26334 tablet_replica.cc:330] T 36014ba00a0847dd89300a82e757d7d8 P 8d738d4e749a48be988ae86c964d91c8: stopping tablet replica
I20260808 18:22:37.527208 26334 raft_consensus.cc:2229] T 36014ba00a0847dd89300a82e757d7d8 P 8d738d4e749a48be988ae86c964d91c8 [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:22:37.527889 26334 raft_consensus.cc:2258] T 36014ba00a0847dd89300a82e757d7d8 P 8d738d4e749a48be988ae86c964d91c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:22:37.528574 26334 ts_tablet_manager.cc:1905] T 36014ba00a0847dd89300a82e757d7d8 P 8d738d4e749a48be988ae86c964d91c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:22:37.530932 26336 tablet_replica.cc:330] T 36014ba00a0847dd89300a82e757d7d8 P 6bd92cb899f44c2cbe8bc5be5655f22f: stopping tablet replica
I20260808 18:22:37.531090 26336 raft_consensus.cc:2229] T 36014ba00a0847dd89300a82e757d7d8 P 6bd92cb899f44c2cbe8bc5be5655f22f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:22:37.531221 26336 raft_consensus.cc:2258] T 36014ba00a0847dd89300a82e757d7d8 P 6bd92cb899f44c2cbe8bc5be5655f22f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:22:37.531491 26335 tablet_replica.cc:330] T 36014ba00a0847dd89300a82e757d7d8 P a4ced4ae4cb143ec86ef7b0a99711d22: stopping tablet replica
I20260808 18:22:37.531683 26335 raft_consensus.cc:2229] T 36014ba00a0847dd89300a82e757d7d8 P a4ced4ae4cb143ec86ef7b0a99711d22 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:22:37.531847 26335 raft_consensus.cc:2258] T 36014ba00a0847dd89300a82e757d7d8 P a4ced4ae4cb143ec86ef7b0a99711d22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:22:37.531916 26334 ts_tablet_manager.cc:1918] T 36014ba00a0847dd89300a82e757d7d8 P 8d738d4e749a48be988ae86c964d91c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 18:22:37.532047 26336 ts_tablet_manager.cc:1905] T 36014ba00a0847dd89300a82e757d7d8 P 6bd92cb899f44c2cbe8bc5be5655f22f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:22:37.532246 26334 log.cc:1196] T 36014ba00a0847dd89300a82e757d7d8 P 8d738d4e749a48be988ae86c964d91c8: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-0/wal/wals/36014ba00a0847dd89300a82e757d7d8
I20260808 18:22:37.532637 26335 ts_tablet_manager.cc:1905] T 36014ba00a0847dd89300a82e757d7d8 P a4ced4ae4cb143ec86ef7b0a99711d22: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:22:37.532662 26334 ts_tablet_manager.cc:1939] T 36014ba00a0847dd89300a82e757d7d8 P 8d738d4e749a48be988ae86c964d91c8: Deleting consensus metadata
I20260808 18:22:37.533326 25686 catalog_manager.cc:4841] TS 8d738d4e749a48be988ae86c964d91c8 (127.23.72.129:33739): tablet 36014ba00a0847dd89300a82e757d7d8 (table test-table [id=1928a4e77cb8411486699a4be2b3f1d4]) successfully deleted
I20260808 18:22:37.535089 26335 ts_tablet_manager.cc:1918] T 36014ba00a0847dd89300a82e757d7d8 P a4ced4ae4cb143ec86ef7b0a99711d22: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 18:22:37.535130 26336 ts_tablet_manager.cc:1918] T 36014ba00a0847dd89300a82e757d7d8 P 6bd92cb899f44c2cbe8bc5be5655f22f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 18:22:37.535189 26335 log.cc:1196] T 36014ba00a0847dd89300a82e757d7d8 P a4ced4ae4cb143ec86ef7b0a99711d22: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-1/wal/wals/36014ba00a0847dd89300a82e757d7d8
I20260808 18:22:37.535207 26336 log.cc:1196] T 36014ba00a0847dd89300a82e757d7d8 P 6bd92cb899f44c2cbe8bc5be5655f22f: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-2/wal/wals/36014ba00a0847dd89300a82e757d7d8
I20260808 18:22:37.535565 26335 ts_tablet_manager.cc:1939] T 36014ba00a0847dd89300a82e757d7d8 P a4ced4ae4cb143ec86ef7b0a99711d22: Deleting consensus metadata
I20260808 18:22:37.535599 26336 ts_tablet_manager.cc:1939] T 36014ba00a0847dd89300a82e757d7d8 P 6bd92cb899f44c2cbe8bc5be5655f22f: Deleting consensus metadata
I20260808 18:22:37.536196 25687 catalog_manager.cc:4841] TS 6bd92cb899f44c2cbe8bc5be5655f22f (127.23.72.131:40149): tablet 36014ba00a0847dd89300a82e757d7d8 (table test-table [id=1928a4e77cb8411486699a4be2b3f1d4]) successfully deleted
I20260808 18:22:37.536373 25687 catalog_manager.cc:4841] TS a4ced4ae4cb143ec86ef7b0a99711d22 (127.23.72.130:36341): tablet 36014ba00a0847dd89300a82e757d7d8 (table test-table [id=1928a4e77cb8411486699a4be2b3f1d4]) successfully deleted
Aug 08 18:22:37 dist-test-slave-5k54 krb5kdc[25652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213357, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 18:22:37 dist-test-slave-5k54 krb5kdc[25652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213357, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:22:37.555555 26346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:37.546888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39122 (local address 127.23.72.190:45825)
0808 18:22:37.547181 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:37.547192 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:37.547257 (+ 65us) server_negotiation.cc:407] Connection header received
0808 18:22:37.547446 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:37.547450 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:37.547505 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:37.547597 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:37.548292 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:37.548854 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:37.549684 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:37.549934 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:37.550042 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:37.552460 (+ 2418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:37.552468 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:37.552470 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:37.552491 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:37.553833 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:37.554253 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:37.554258 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:37.554259 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:37.554350 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:37.554620 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:37.554625 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:37.554626 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:37.554802 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:37.554954 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:37.555182 (+ 228us) server_negotiation.cc:299] Negotiation successful
0808 18:22:37.555302 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":108,"threads_started":1}
I20260808 18:22:37.558061 25704 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39122:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:22:37.558502 25704 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:22:37.563268 26001 tablet_service.cc:1467] Processing CreateTablet for tablet 758adfc31c5648aab6148884d9a237c6 (DEFAULT_TABLE table=test-table [id=85fbf38bff2542c7a4573c7920683d62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:22:37.563339 25852 tablet_service.cc:1467] Processing CreateTablet for tablet 758adfc31c5648aab6148884d9a237c6 (DEFAULT_TABLE table=test-table [id=85fbf38bff2542c7a4573c7920683d62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:22:37.563397 26153 tablet_service.cc:1467] Processing CreateTablet for tablet 758adfc31c5648aab6148884d9a237c6 (DEFAULT_TABLE table=test-table [id=85fbf38bff2542c7a4573c7920683d62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:22:37.563632 26001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 758adfc31c5648aab6148884d9a237c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:37.563637 25852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 758adfc31c5648aab6148884d9a237c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:37.563741 26153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 758adfc31c5648aab6148884d9a237c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:37.566313 26347 tablet_bootstrap.cc:492] T 758adfc31c5648aab6148884d9a237c6 P 6bd92cb899f44c2cbe8bc5be5655f22f: Bootstrap starting.
I20260808 18:22:37.566673 26349 tablet_bootstrap.cc:492] T 758adfc31c5648aab6148884d9a237c6 P 8d738d4e749a48be988ae86c964d91c8: Bootstrap starting.
I20260808 18:22:37.567224 26347 tablet_bootstrap.cc:654] T 758adfc31c5648aab6148884d9a237c6 P 6bd92cb899f44c2cbe8bc5be5655f22f: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:37.567551 26349 tablet_bootstrap.cc:654] T 758adfc31c5648aab6148884d9a237c6 P 8d738d4e749a48be988ae86c964d91c8: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:37.568259 26348 tablet_bootstrap.cc:492] T 758adfc31c5648aab6148884d9a237c6 P a4ced4ae4cb143ec86ef7b0a99711d22: Bootstrap starting.
I20260808 18:22:37.568425 26347 tablet_bootstrap.cc:492] T 758adfc31c5648aab6148884d9a237c6 P 6bd92cb899f44c2cbe8bc5be5655f22f: No bootstrap required, opened a new log
I20260808 18:22:37.568517 26347 ts_tablet_manager.cc:1397] T 758adfc31c5648aab6148884d9a237c6 P 6bd92cb899f44c2cbe8bc5be5655f22f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 18:22:37.569036 26349 tablet_bootstrap.cc:492] T 758adfc31c5648aab6148884d9a237c6 P 8d738d4e749a48be988ae86c964d91c8: No bootstrap required, opened a new log
I20260808 18:22:37.569036 26347 raft_consensus.cc:348] T 758adfc31c5648aab6148884d9a237c6 P 6bd92cb899f44c2cbe8bc5be5655f22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } }
I20260808 18:22:37.569200 26348 tablet_bootstrap.cc:654] T 758adfc31c5648aab6148884d9a237c6 P a4ced4ae4cb143ec86ef7b0a99711d22: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:37.569218 26349 ts_tablet_manager.cc:1397] T 758adfc31c5648aab6148884d9a237c6 P 8d738d4e749a48be988ae86c964d91c8: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260808 18:22:37.569250 26347 raft_consensus.cc:374] T 758adfc31c5648aab6148884d9a237c6 P 6bd92cb899f44c2cbe8bc5be5655f22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:37.569379 26347 raft_consensus.cc:729] T 758adfc31c5648aab6148884d9a237c6 P 6bd92cb899f44c2cbe8bc5be5655f22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bd92cb899f44c2cbe8bc5be5655f22f, State: Initialized, Role: FOLLOWER
I20260808 18:22:37.569555 26347 consensus_queue.cc:260] T 758adfc31c5648aab6148884d9a237c6 P 6bd92cb899f44c2cbe8bc5be5655f22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } }
I20260808 18:22:37.569693 26349 raft_consensus.cc:348] T 758adfc31c5648aab6148884d9a237c6 P 8d738d4e749a48be988ae86c964d91c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } }
I20260808 18:22:37.569900 26349 raft_consensus.cc:374] T 758adfc31c5648aab6148884d9a237c6 P 8d738d4e749a48be988ae86c964d91c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:37.569952 26347 ts_tablet_manager.cc:1428] T 758adfc31c5648aab6148884d9a237c6 P 6bd92cb899f44c2cbe8bc5be5655f22f: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 18:22:37.569998 26349 raft_consensus.cc:729] T 758adfc31c5648aab6148884d9a237c6 P 8d738d4e749a48be988ae86c964d91c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d738d4e749a48be988ae86c964d91c8, State: Initialized, Role: FOLLOWER
I20260808 18:22:37.570243 26349 consensus_queue.cc:260] T 758adfc31c5648aab6148884d9a237c6 P 8d738d4e749a48be988ae86c964d91c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } }
I20260808 18:22:37.570561 26348 tablet_bootstrap.cc:492] T 758adfc31c5648aab6148884d9a237c6 P a4ced4ae4cb143ec86ef7b0a99711d22: No bootstrap required, opened a new log
I20260808 18:22:37.570649 26348 ts_tablet_manager.cc:1397] T 758adfc31c5648aab6148884d9a237c6 P a4ced4ae4cb143ec86ef7b0a99711d22: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 18:22:37.570636 26349 ts_tablet_manager.cc:1428] T 758adfc31c5648aab6148884d9a237c6 P 8d738d4e749a48be988ae86c964d91c8: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 18:22:37.571044 26348 raft_consensus.cc:348] T 758adfc31c5648aab6148884d9a237c6 P a4ced4ae4cb143ec86ef7b0a99711d22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } }
I20260808 18:22:37.571159 26348 raft_consensus.cc:374] T 758adfc31c5648aab6148884d9a237c6 P a4ced4ae4cb143ec86ef7b0a99711d22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:37.571190 26348 raft_consensus.cc:729] T 758adfc31c5648aab6148884d9a237c6 P a4ced4ae4cb143ec86ef7b0a99711d22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4ced4ae4cb143ec86ef7b0a99711d22, State: Initialized, Role: FOLLOWER
I20260808 18:22:37.571314 26348 consensus_queue.cc:260] T 758adfc31c5648aab6148884d9a237c6 P a4ced4ae4cb143ec86ef7b0a99711d22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } }
I20260808 18:22:37.571708 26348 ts_tablet_manager.cc:1428] T 758adfc31c5648aab6148884d9a237c6 P a4ced4ae4cb143ec86ef7b0a99711d22: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:22:37.903301 26321 raft_consensus.cc:482] T 758adfc31c5648aab6148884d9a237c6 P 8d738d4e749a48be988ae86c964d91c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:22:37.903477 26321 raft_consensus.cc:504] T 758adfc31c5648aab6148884d9a237c6 P 8d738d4e749a48be988ae86c964d91c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } }
I20260808 18:22:37.903955 26321 leader_election.cc:288] T 758adfc31c5648aab6148884d9a237c6 P 8d738d4e749a48be988ae86c964d91c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a4ced4ae4cb143ec86ef7b0a99711d22 (127.23.72.130:36341), 6bd92cb899f44c2cbe8bc5be5655f22f (127.23.72.131:40149)
I20260808 18:22:37.904469 26022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "758adfc31c5648aab6148884d9a237c6" candidate_uuid: "8d738d4e749a48be988ae86c964d91c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" is_pre_election: true
I20260808 18:22:37.904525 26175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "758adfc31c5648aab6148884d9a237c6" candidate_uuid: "8d738d4e749a48be988ae86c964d91c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" is_pre_election: true
I20260808 18:22:37.904627 26022 raft_consensus.cc:2454] T 758adfc31c5648aab6148884d9a237c6 P a4ced4ae4cb143ec86ef7b0a99711d22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d738d4e749a48be988ae86c964d91c8 in term 0.
I20260808 18:22:37.904706 26175 raft_consensus.cc:2454] T 758adfc31c5648aab6148884d9a237c6 P 6bd92cb899f44c2cbe8bc5be5655f22f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d738d4e749a48be988ae86c964d91c8 in term 0.
I20260808 18:22:37.904994 25798 leader_election.cc:302] T 758adfc31c5648aab6148884d9a237c6 P 8d738d4e749a48be988ae86c964d91c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d738d4e749a48be988ae86c964d91c8, a4ced4ae4cb143ec86ef7b0a99711d22; no voters:
I20260808 18:22:37.905277 26321 raft_consensus.cc:2781] T 758adfc31c5648aab6148884d9a237c6 P 8d738d4e749a48be988ae86c964d91c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:22:37.905364 26321 raft_consensus.cc:482] T 758adfc31c5648aab6148884d9a237c6 P 8d738d4e749a48be988ae86c964d91c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:22:37.905417 26321 raft_consensus.cc:3037] T 758adfc31c5648aab6148884d9a237c6 P 8d738d4e749a48be988ae86c964d91c8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:37.906440 26321 raft_consensus.cc:504] T 758adfc31c5648aab6148884d9a237c6 P 8d738d4e749a48be988ae86c964d91c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } }
I20260808 18:22:37.906777 26321 leader_election.cc:288] T 758adfc31c5648aab6148884d9a237c6 P 8d738d4e749a48be988ae86c964d91c8 [CANDIDATE]: Term 1 election: Requested vote from peers a4ced4ae4cb143ec86ef7b0a99711d22 (127.23.72.130:36341), 6bd92cb899f44c2cbe8bc5be5655f22f (127.23.72.131:40149)
I20260808 18:22:37.907207 26022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "758adfc31c5648aab6148884d9a237c6" candidate_uuid: "8d738d4e749a48be988ae86c964d91c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22"
I20260808 18:22:37.907236 26175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "758adfc31c5648aab6148884d9a237c6" candidate_uuid: "8d738d4e749a48be988ae86c964d91c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f"
I20260808 18:22:37.907344 26175 raft_consensus.cc:3037] T 758adfc31c5648aab6148884d9a237c6 P 6bd92cb899f44c2cbe8bc5be5655f22f [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:37.907344 26022 raft_consensus.cc:3037] T 758adfc31c5648aab6148884d9a237c6 P a4ced4ae4cb143ec86ef7b0a99711d22 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:37.908543 26175 raft_consensus.cc:2454] T 758adfc31c5648aab6148884d9a237c6 P 6bd92cb899f44c2cbe8bc5be5655f22f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d738d4e749a48be988ae86c964d91c8 in term 1.
I20260808 18:22:37.908576 26022 raft_consensus.cc:2454] T 758adfc31c5648aab6148884d9a237c6 P a4ced4ae4cb143ec86ef7b0a99711d22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d738d4e749a48be988ae86c964d91c8 in term 1.
I20260808 18:22:37.908838 25798 leader_election.cc:302] T 758adfc31c5648aab6148884d9a237c6 P 8d738d4e749a48be988ae86c964d91c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bd92cb899f44c2cbe8bc5be5655f22f, 8d738d4e749a48be988ae86c964d91c8; no voters:
I20260808 18:22:37.909018 26321 raft_consensus.cc:2781] T 758adfc31c5648aab6148884d9a237c6 P 8d738d4e749a48be988ae86c964d91c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:22:37.909109 26321 raft_consensus.cc:686] T 758adfc31c5648aab6148884d9a237c6 P 8d738d4e749a48be988ae86c964d91c8 [term 1 LEADER]: Becoming Leader. State: Replica: 8d738d4e749a48be988ae86c964d91c8, State: Running, Role: LEADER
I20260808 18:22:37.909281 26321 consensus_queue.cc:237] T 758adfc31c5648aab6148884d9a237c6 P 8d738d4e749a48be988ae86c964d91c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } }
I20260808 18:22:37.911141 25701 catalog_manager.cc:5495] T 758adfc31c5648aab6148884d9a237c6 P 8d738d4e749a48be988ae86c964d91c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8d738d4e749a48be988ae86c964d91c8 (127.23.72.129). New cstate: current_term: 1 leader_uuid: "8d738d4e749a48be988ae86c964d91c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } health_report { overall_health: UNKNOWN } } }
I20260808 18:22:37.937354 26358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:37.934182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46086 (local address 127.23.72.129:33739)
0808 18:22:37.934451 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:37.934458 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:37.934469 (+ 11us) server_negotiation.cc:407] Connection header received
0808 18:22:37.934506 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:37.934508 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:37.934546 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:37.934615 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:22:37.934914 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:37.935502 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:37.936338 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:37.936564 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:37.936643 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:37.936925 (+ 282us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:22:37.936992 (+ 67us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:22:37.937045 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:37.937135 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 18:22:37.937177 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":130,"threads_started":1}
I20260808 18:22:37.943130 26175 raft_consensus.cc:1261] T 758adfc31c5648aab6148884d9a237c6 P 6bd92cb899f44c2cbe8bc5be5655f22f [term 1 FOLLOWER]: Refusing update from remote peer 8d738d4e749a48be988ae86c964d91c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:22:37.943181 26022 raft_consensus.cc:1261] T 758adfc31c5648aab6148884d9a237c6 P a4ced4ae4cb143ec86ef7b0a99711d22 [term 1 FOLLOWER]: Refusing update from remote peer 8d738d4e749a48be988ae86c964d91c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:22:37.943583 26316 consensus_queue.cc:1035] T 758adfc31c5648aab6148884d9a237c6 P 8d738d4e749a48be988ae86c964d91c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:22:37.943753 26288 consensus_queue.cc:1035] T 758adfc31c5648aab6148884d9a237c6 P 8d738d4e749a48be988ae86c964d91c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:22:37.973347 25701 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39122:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:22:37.973517 25701 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39122:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:22:37.975348 25701 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9d816eb59738467ebf0d0d6cbf7bbcaf: Sending DeleteTablet for 3 replicas of tablet 758adfc31c5648aab6148884d9a237c6
I20260808 18:22:37.975874 25852 tablet_service.cc:1514] Processing DeleteTablet for tablet 758adfc31c5648aab6148884d9a237c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:22:37 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:46058
I20260808 18:22:37.976044 26001 tablet_service.cc:1514] Processing DeleteTablet for tablet 758adfc31c5648aab6148884d9a237c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:22:37 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:52570
I20260808 18:22:37.976063 26153 tablet_service.cc:1514] Processing DeleteTablet for tablet 758adfc31c5648aab6148884d9a237c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:22:37 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:35138
I20260808 18:22:37.976241 26335 tablet_replica.cc:330] T 758adfc31c5648aab6148884d9a237c6 P a4ced4ae4cb143ec86ef7b0a99711d22: stopping tablet replica
I20260808 18:22:37.976243 26334 tablet_replica.cc:330] T 758adfc31c5648aab6148884d9a237c6 P 8d738d4e749a48be988ae86c964d91c8: stopping tablet replica
I20260808 18:22:37.976258 26336 tablet_replica.cc:330] T 758adfc31c5648aab6148884d9a237c6 P 6bd92cb899f44c2cbe8bc5be5655f22f: stopping tablet replica
I20260808 18:22:37.976358 26334 raft_consensus.cc:2229] T 758adfc31c5648aab6148884d9a237c6 P 8d738d4e749a48be988ae86c964d91c8 [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:22:37.976370 26335 raft_consensus.cc:2229] T 758adfc31c5648aab6148884d9a237c6 P a4ced4ae4cb143ec86ef7b0a99711d22 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:22:37.976382 26336 raft_consensus.cc:2229] T 758adfc31c5648aab6148884d9a237c6 P 6bd92cb899f44c2cbe8bc5be5655f22f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:22:37.976480 26335 raft_consensus.cc:2258] T 758adfc31c5648aab6148884d9a237c6 P a4ced4ae4cb143ec86ef7b0a99711d22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:22:37.976545 26336 raft_consensus.cc:2258] T 758adfc31c5648aab6148884d9a237c6 P 6bd92cb899f44c2cbe8bc5be5655f22f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:22:37.976549 26334 raft_consensus.cc:2258] T 758adfc31c5648aab6148884d9a237c6 P 8d738d4e749a48be988ae86c964d91c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:22:37.977169 26335 ts_tablet_manager.cc:1905] T 758adfc31c5648aab6148884d9a237c6 P a4ced4ae4cb143ec86ef7b0a99711d22: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:22:37.977180 26336 ts_tablet_manager.cc:1905] T 758adfc31c5648aab6148884d9a237c6 P 6bd92cb899f44c2cbe8bc5be5655f22f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:22:37.977519 25701 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39122:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:22:37.977895 25701 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:22:37.979770 26336 ts_tablet_manager.cc:1918] T 758adfc31c5648aab6148884d9a237c6 P 6bd92cb899f44c2cbe8bc5be5655f22f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 18:22:37.979866 26336 log.cc:1196] T 758adfc31c5648aab6148884d9a237c6 P 6bd92cb899f44c2cbe8bc5be5655f22f: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-2/wal/wals/758adfc31c5648aab6148884d9a237c6
I20260808 18:22:37.979883 26335 ts_tablet_manager.cc:1918] T 758adfc31c5648aab6148884d9a237c6 P a4ced4ae4cb143ec86ef7b0a99711d22: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 18:22:37.979964 26335 log.cc:1196] T 758adfc31c5648aab6148884d9a237c6 P a4ced4ae4cb143ec86ef7b0a99711d22: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-1/wal/wals/758adfc31c5648aab6148884d9a237c6
I20260808 18:22:37.980171 26336 ts_tablet_manager.cc:1939] T 758adfc31c5648aab6148884d9a237c6 P 6bd92cb899f44c2cbe8bc5be5655f22f: Deleting consensus metadata
I20260808 18:22:37.980204 26335 ts_tablet_manager.cc:1939] T 758adfc31c5648aab6148884d9a237c6 P a4ced4ae4cb143ec86ef7b0a99711d22: Deleting consensus metadata
I20260808 18:22:37.980873 25687 catalog_manager.cc:4841] TS 6bd92cb899f44c2cbe8bc5be5655f22f (127.23.72.131:40149): tablet 758adfc31c5648aab6148884d9a237c6 (table test-table [id=85fbf38bff2542c7a4573c7920683d62]) successfully deleted
I20260808 18:22:37.981283 26334 ts_tablet_manager.cc:1905] T 758adfc31c5648aab6148884d9a237c6 P 8d738d4e749a48be988ae86c964d91c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:22:37.981654 25687 catalog_manager.cc:4841] TS a4ced4ae4cb143ec86ef7b0a99711d22 (127.23.72.130:36341): tablet 758adfc31c5648aab6148884d9a237c6 (table test-table [id=85fbf38bff2542c7a4573c7920683d62]) successfully deleted
I20260808 18:22:37.982880 25852 tablet_service.cc:1467] Processing CreateTablet for tablet 8d022fad6bcf49018a4de921c136b961 (DEFAULT_TABLE table=test-table [id=5992f0faf9e7424fa7aff95e234f6ee6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:22:37.982878 26001 tablet_service.cc:1467] Processing CreateTablet for tablet 8d022fad6bcf49018a4de921c136b961 (DEFAULT_TABLE table=test-table [id=5992f0faf9e7424fa7aff95e234f6ee6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:22:37.983120 25852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d022fad6bcf49018a4de921c136b961. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:37.983119 26001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d022fad6bcf49018a4de921c136b961. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:37.983779 26334 ts_tablet_manager.cc:1918] T 758adfc31c5648aab6148884d9a237c6 P 8d738d4e749a48be988ae86c964d91c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 18:22:37.983880 26334 log.cc:1196] T 758adfc31c5648aab6148884d9a237c6 P 8d738d4e749a48be988ae86c964d91c8: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-0/wal/wals/758adfc31c5648aab6148884d9a237c6
I20260808 18:22:37.984115 26334 ts_tablet_manager.cc:1939] T 758adfc31c5648aab6148884d9a237c6 P 8d738d4e749a48be988ae86c964d91c8: Deleting consensus metadata
I20260808 18:22:37.984977 25686 catalog_manager.cc:4841] TS 8d738d4e749a48be988ae86c964d91c8 (127.23.72.129:33739): tablet 758adfc31c5648aab6148884d9a237c6 (table test-table [id=85fbf38bff2542c7a4573c7920683d62]) successfully deleted
I20260808 18:22:37.985272 26349 tablet_bootstrap.cc:492] T 8d022fad6bcf49018a4de921c136b961 P 8d738d4e749a48be988ae86c964d91c8: Bootstrap starting.
I20260808 18:22:37.985312 26348 tablet_bootstrap.cc:492] T 8d022fad6bcf49018a4de921c136b961 P a4ced4ae4cb143ec86ef7b0a99711d22: Bootstrap starting.
I20260808 18:22:37.986097 26349 tablet_bootstrap.cc:654] T 8d022fad6bcf49018a4de921c136b961 P 8d738d4e749a48be988ae86c964d91c8: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:37.986107 26348 tablet_bootstrap.cc:654] T 8d022fad6bcf49018a4de921c136b961 P a4ced4ae4cb143ec86ef7b0a99711d22: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:37.986985 26153 tablet_service.cc:1467] Processing CreateTablet for tablet 8d022fad6bcf49018a4de921c136b961 (DEFAULT_TABLE table=test-table [id=5992f0faf9e7424fa7aff95e234f6ee6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:22:37.987061 26349 tablet_bootstrap.cc:492] T 8d022fad6bcf49018a4de921c136b961 P 8d738d4e749a48be988ae86c964d91c8: No bootstrap required, opened a new log
I20260808 18:22:37.987118 26349 ts_tablet_manager.cc:1397] T 8d022fad6bcf49018a4de921c136b961 P 8d738d4e749a48be988ae86c964d91c8: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 18:22:37.987160 26348 tablet_bootstrap.cc:492] T 8d022fad6bcf49018a4de921c136b961 P a4ced4ae4cb143ec86ef7b0a99711d22: No bootstrap required, opened a new log
I20260808 18:22:37.987233 26153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d022fad6bcf49018a4de921c136b961. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:37.987234 26348 ts_tablet_manager.cc:1397] T 8d022fad6bcf49018a4de921c136b961 P a4ced4ae4cb143ec86ef7b0a99711d22: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 18:22:37.987386 26349 raft_consensus.cc:348] T 8d022fad6bcf49018a4de921c136b961 P 8d738d4e749a48be988ae86c964d91c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } }
I20260808 18:22:37.987537 26349 raft_consensus.cc:374] T 8d022fad6bcf49018a4de921c136b961 P 8d738d4e749a48be988ae86c964d91c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:37.987588 26349 raft_consensus.cc:729] T 8d022fad6bcf49018a4de921c136b961 P 8d738d4e749a48be988ae86c964d91c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d738d4e749a48be988ae86c964d91c8, State: Initialized, Role: FOLLOWER
I20260808 18:22:37.987674 26348 raft_consensus.cc:348] T 8d022fad6bcf49018a4de921c136b961 P a4ced4ae4cb143ec86ef7b0a99711d22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } }
I20260808 18:22:37.987768 26348 raft_consensus.cc:374] T 8d022fad6bcf49018a4de921c136b961 P a4ced4ae4cb143ec86ef7b0a99711d22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:37.987725 26349 consensus_queue.cc:260] T 8d022fad6bcf49018a4de921c136b961 P 8d738d4e749a48be988ae86c964d91c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } }
I20260808 18:22:37.987797 26348 raft_consensus.cc:729] T 8d022fad6bcf49018a4de921c136b961 P a4ced4ae4cb143ec86ef7b0a99711d22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4ced4ae4cb143ec86ef7b0a99711d22, State: Initialized, Role: FOLLOWER
I20260808 18:22:37.987908 26348 consensus_queue.cc:260] T 8d022fad6bcf49018a4de921c136b961 P a4ced4ae4cb143ec86ef7b0a99711d22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } }
I20260808 18:22:37.988024 26349 ts_tablet_manager.cc:1428] T 8d022fad6bcf49018a4de921c136b961 P 8d738d4e749a48be988ae86c964d91c8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:22:37.988132 26348 ts_tablet_manager.cc:1428] T 8d022fad6bcf49018a4de921c136b961 P a4ced4ae4cb143ec86ef7b0a99711d22: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:22:37.989470 26347 tablet_bootstrap.cc:492] T 8d022fad6bcf49018a4de921c136b961 P 6bd92cb899f44c2cbe8bc5be5655f22f: Bootstrap starting.
I20260808 18:22:37.990281 26347 tablet_bootstrap.cc:654] T 8d022fad6bcf49018a4de921c136b961 P 6bd92cb899f44c2cbe8bc5be5655f22f: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:37.991410 26347 tablet_bootstrap.cc:492] T 8d022fad6bcf49018a4de921c136b961 P 6bd92cb899f44c2cbe8bc5be5655f22f: No bootstrap required, opened a new log
I20260808 18:22:37.991557 26347 ts_tablet_manager.cc:1397] T 8d022fad6bcf49018a4de921c136b961 P 6bd92cb899f44c2cbe8bc5be5655f22f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 18:22:37.991997 26347 raft_consensus.cc:348] T 8d022fad6bcf49018a4de921c136b961 P 6bd92cb899f44c2cbe8bc5be5655f22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } }
I20260808 18:22:37.992105 26347 raft_consensus.cc:374] T 8d022fad6bcf49018a4de921c136b961 P 6bd92cb899f44c2cbe8bc5be5655f22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:37.992149 26347 raft_consensus.cc:729] T 8d022fad6bcf49018a4de921c136b961 P 6bd92cb899f44c2cbe8bc5be5655f22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bd92cb899f44c2cbe8bc5be5655f22f, State: Initialized, Role: FOLLOWER
I20260808 18:22:37.992280 26347 consensus_queue.cc:260] T 8d022fad6bcf49018a4de921c136b961 P 6bd92cb899f44c2cbe8bc5be5655f22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } }
I20260808 18:22:37.992540 26347 ts_tablet_manager.cc:1428] T 8d022fad6bcf49018a4de921c136b961 P 6bd92cb899f44c2cbe8bc5be5655f22f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 18:22:38.003979 26316 raft_consensus.cc:482] T 8d022fad6bcf49018a4de921c136b961 P 8d738d4e749a48be988ae86c964d91c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:22:38.004081 26316 raft_consensus.cc:504] T 8d022fad6bcf49018a4de921c136b961 P 8d738d4e749a48be988ae86c964d91c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } }
I20260808 18:22:38.004428 26316 leader_election.cc:288] T 8d022fad6bcf49018a4de921c136b961 P 8d738d4e749a48be988ae86c964d91c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a4ced4ae4cb143ec86ef7b0a99711d22 (127.23.72.130:36341), 6bd92cb899f44c2cbe8bc5be5655f22f (127.23.72.131:40149)
I20260808 18:22:38.004834 26022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d022fad6bcf49018a4de921c136b961" candidate_uuid: "8d738d4e749a48be988ae86c964d91c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" is_pre_election: true
I20260808 18:22:38.004866 26175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d022fad6bcf49018a4de921c136b961" candidate_uuid: "8d738d4e749a48be988ae86c964d91c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" is_pre_election: true
I20260808 18:22:38.004980 26175 raft_consensus.cc:2454] T 8d022fad6bcf49018a4de921c136b961 P 6bd92cb899f44c2cbe8bc5be5655f22f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d738d4e749a48be988ae86c964d91c8 in term 0.
I20260808 18:22:38.004980 26022 raft_consensus.cc:2454] T 8d022fad6bcf49018a4de921c136b961 P a4ced4ae4cb143ec86ef7b0a99711d22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d738d4e749a48be988ae86c964d91c8 in term 0.
I20260808 18:22:38.005277 25798 leader_election.cc:302] T 8d022fad6bcf49018a4de921c136b961 P 8d738d4e749a48be988ae86c964d91c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bd92cb899f44c2cbe8bc5be5655f22f, 8d738d4e749a48be988ae86c964d91c8; no voters:
I20260808 18:22:38.005466 26316 raft_consensus.cc:2781] T 8d022fad6bcf49018a4de921c136b961 P 8d738d4e749a48be988ae86c964d91c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:22:38.005529 26316 raft_consensus.cc:482] T 8d022fad6bcf49018a4de921c136b961 P 8d738d4e749a48be988ae86c964d91c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:22:38.005571 26316 raft_consensus.cc:3037] T 8d022fad6bcf49018a4de921c136b961 P 8d738d4e749a48be988ae86c964d91c8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:38.006538 26316 raft_consensus.cc:504] T 8d022fad6bcf49018a4de921c136b961 P 8d738d4e749a48be988ae86c964d91c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } }
I20260808 18:22:38.006923 26316 leader_election.cc:288] T 8d022fad6bcf49018a4de921c136b961 P 8d738d4e749a48be988ae86c964d91c8 [CANDIDATE]: Term 1 election: Requested vote from peers a4ced4ae4cb143ec86ef7b0a99711d22 (127.23.72.130:36341), 6bd92cb899f44c2cbe8bc5be5655f22f (127.23.72.131:40149)
I20260808 18:22:38.007208 26022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d022fad6bcf49018a4de921c136b961" candidate_uuid: "8d738d4e749a48be988ae86c964d91c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22"
I20260808 18:22:38.007262 26175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d022fad6bcf49018a4de921c136b961" candidate_uuid: "8d738d4e749a48be988ae86c964d91c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f"
I20260808 18:22:38.007344 26022 raft_consensus.cc:3037] T 8d022fad6bcf49018a4de921c136b961 P a4ced4ae4cb143ec86ef7b0a99711d22 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:38.007380 26175 raft_consensus.cc:3037] T 8d022fad6bcf49018a4de921c136b961 P 6bd92cb899f44c2cbe8bc5be5655f22f [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:38.008226 26022 raft_consensus.cc:2454] T 8d022fad6bcf49018a4de921c136b961 P a4ced4ae4cb143ec86ef7b0a99711d22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d738d4e749a48be988ae86c964d91c8 in term 1.
I20260808 18:22:38.008279 26175 raft_consensus.cc:2454] T 8d022fad6bcf49018a4de921c136b961 P 6bd92cb899f44c2cbe8bc5be5655f22f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d738d4e749a48be988ae86c964d91c8 in term 1.
I20260808 18:22:38.008589 25798 leader_election.cc:302] T 8d022fad6bcf49018a4de921c136b961 P 8d738d4e749a48be988ae86c964d91c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bd92cb899f44c2cbe8bc5be5655f22f, 8d738d4e749a48be988ae86c964d91c8; no voters:
I20260808 18:22:38.008750 26316 raft_consensus.cc:2781] T 8d022fad6bcf49018a4de921c136b961 P 8d738d4e749a48be988ae86c964d91c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:22:38.008852 26316 raft_consensus.cc:686] T 8d022fad6bcf49018a4de921c136b961 P 8d738d4e749a48be988ae86c964d91c8 [term 1 LEADER]: Becoming Leader. State: Replica: 8d738d4e749a48be988ae86c964d91c8, State: Running, Role: LEADER
I20260808 18:22:38.009001 26316 consensus_queue.cc:237] T 8d022fad6bcf49018a4de921c136b961 P 8d738d4e749a48be988ae86c964d91c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } }
I20260808 18:22:38.010473 25701 catalog_manager.cc:5495] T 8d022fad6bcf49018a4de921c136b961 P 8d738d4e749a48be988ae86c964d91c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8d738d4e749a48be988ae86c964d91c8 (127.23.72.129). New cstate: current_term: 1 leader_uuid: "8d738d4e749a48be988ae86c964d91c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d738d4e749a48be988ae86c964d91c8" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 33739 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 } health_report { overall_health: UNKNOWN } } }
I20260808 18:22:38.025478 26022 raft_consensus.cc:1261] T 8d022fad6bcf49018a4de921c136b961 P a4ced4ae4cb143ec86ef7b0a99711d22 [term 1 FOLLOWER]: Refusing update from remote peer 8d738d4e749a48be988ae86c964d91c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:22:38.025522 26175 raft_consensus.cc:1261] T 8d022fad6bcf49018a4de921c136b961 P 6bd92cb899f44c2cbe8bc5be5655f22f [term 1 FOLLOWER]: Refusing update from remote peer 8d738d4e749a48be988ae86c964d91c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:22:38.025864 26321 consensus_queue.cc:1035] T 8d022fad6bcf49018a4de921c136b961 P 8d738d4e749a48be988ae86c964d91c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4ced4ae4cb143ec86ef7b0a99711d22" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 36341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:22:38.026012 26363 consensus_queue.cc:1035] T 8d022fad6bcf49018a4de921c136b961 P 8d738d4e749a48be988ae86c964d91c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bd92cb899f44c2cbe8bc5be5655f22f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 18:22:38.039196 26314 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 18:22:39.586138 25702 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39122:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:22:39.586284 25702 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39122:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:22:39.588760 25702 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9d816eb59738467ebf0d0d6cbf7bbcaf: Sending DeleteTablet for 3 replicas of tablet 8d022fad6bcf49018a4de921c136b961
I20260808 18:22:39.589315 25849 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d022fad6bcf49018a4de921c136b961 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:22:39 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:46058
I20260808 18:22:39.589525 26153 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d022fad6bcf49018a4de921c136b961 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:22:39 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:35138
I20260808 18:22:39.589663 26001 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d022fad6bcf49018a4de921c136b961 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:22:39 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:52570
I20260808 18:22:39.589708 26394 tablet_replica.cc:330] T 8d022fad6bcf49018a4de921c136b961 P 8d738d4e749a48be988ae86c964d91c8: stopping tablet replica
I20260808 18:22:39.589859 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 25781
I20260808 18:22:39.595196 26396 tablet_replica.cc:330] T 8d022fad6bcf49018a4de921c136b961 P a4ced4ae4cb143ec86ef7b0a99711d22: stopping tablet replica
I20260808 18:22:39.595387 26396 raft_consensus.cc:2229] T 8d022fad6bcf49018a4de921c136b961 P a4ced4ae4cb143ec86ef7b0a99711d22 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:22:39.595508 26395 tablet_replica.cc:330] T 8d022fad6bcf49018a4de921c136b961 P 6bd92cb899f44c2cbe8bc5be5655f22f: stopping tablet replica
I20260808 18:22:39.595614 26396 raft_consensus.cc:2258] T 8d022fad6bcf49018a4de921c136b961 P a4ced4ae4cb143ec86ef7b0a99711d22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:22:39.595685 26395 raft_consensus.cc:2229] T 8d022fad6bcf49018a4de921c136b961 P 6bd92cb899f44c2cbe8bc5be5655f22f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:22:39.595854 26395 raft_consensus.cc:2258] T 8d022fad6bcf49018a4de921c136b961 P 6bd92cb899f44c2cbe8bc5be5655f22f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:22:39.596418 26396 ts_tablet_manager.cc:1905] T 8d022fad6bcf49018a4de921c136b961 P a4ced4ae4cb143ec86ef7b0a99711d22: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:22:39.596510 26395 ts_tablet_manager.cc:1905] T 8d022fad6bcf49018a4de921c136b961 P 6bd92cb899f44c2cbe8bc5be5655f22f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:22:39.599092 26395 ts_tablet_manager.cc:1918] T 8d022fad6bcf49018a4de921c136b961 P 6bd92cb899f44c2cbe8bc5be5655f22f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 18:22:39.599092 26396 ts_tablet_manager.cc:1918] T 8d022fad6bcf49018a4de921c136b961 P a4ced4ae4cb143ec86ef7b0a99711d22: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 18:22:39.599174 26396 log.cc:1196] T 8d022fad6bcf49018a4de921c136b961 P a4ced4ae4cb143ec86ef7b0a99711d22: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-1/wal/wals/8d022fad6bcf49018a4de921c136b961
I20260808 18:22:39.599246 26395 log.cc:1196] T 8d022fad6bcf49018a4de921c136b961 P 6bd92cb899f44c2cbe8bc5be5655f22f: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213345431840-23842-0/minicluster-data/ts-2/wal/wals/8d022fad6bcf49018a4de921c136b961
I20260808 18:22:39.599579 26396 ts_tablet_manager.cc:1939] T 8d022fad6bcf49018a4de921c136b961 P a4ced4ae4cb143ec86ef7b0a99711d22: Deleting consensus metadata
I20260808 18:22:39.599594 26395 ts_tablet_manager.cc:1939] T 8d022fad6bcf49018a4de921c136b961 P 6bd92cb899f44c2cbe8bc5be5655f22f: Deleting consensus metadata
I20260808 18:22:39.600306 25687 catalog_manager.cc:4841] TS 6bd92cb899f44c2cbe8bc5be5655f22f (127.23.72.131:40149): tablet 8d022fad6bcf49018a4de921c136b961 (table test-table [id=5992f0faf9e7424fa7aff95e234f6ee6]) successfully deleted
I20260808 18:22:39.600497 25687 catalog_manager.cc:4841] TS a4ced4ae4cb143ec86ef7b0a99711d22 (127.23.72.130:36341): tablet 8d022fad6bcf49018a4de921c136b961 (table test-table [id=5992f0faf9e7424fa7aff95e234f6ee6]) successfully deleted
W20260808 18:22:39.601670 25686 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.72.129:33739 (error 108)
I20260808 18:22:39.602442 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 25932
I20260808 18:22:39.605400 26398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:39.602419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:33739 (local address 127.0.0.1:46096)
0808 18:22:39.605223 (+ 2804us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:39.605304 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.129:33739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2726,"thread_start_us":84,"threads_started":1}
W20260808 18:22:39.605506 25686 catalog_manager.cc:4584] TS 8d738d4e749a48be988ae86c964d91c8 (127.23.72.129:33739): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8d022fad6bcf49018a4de921c136b961: Network error: Client connection negotiation failed: client connection to 127.23.72.129:33739: connect: Connection refused (error 111)
I20260808 18:22:39.611362 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 26081
I20260808 18:22:39.620198 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 25667
2026-08-08T18:22:39Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6449 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:22:39 dist-test-slave-5k54 krb5kdc[26402](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:22:39 dist-test-slave-5k54 krb5kdc[26402](info): set up 2 sockets
Aug 08 18:22:39 dist-test-slave-5k54 krb5kdc[26402](info): commencing operation
krb5kdc: starting...
W20260808 18:22:40.705024 23842 mini_kdc.cc:121] Time spent starting KDC: real 1.059s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Aug 08 18:22:40 dist-test-slave-5k54 krb5kdc[26402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213360, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:22:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:22:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.72.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
I20260808 18:22:40.835379 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:46465
--webserver_interface=127.23.72.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:39891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:46465
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:40.946775 26417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:40.947064 26417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:40.947105 26417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:40.951064 26417 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:22:40.951118 26417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:40.951138 26417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:40.951153 26417 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:22:40.951210 26417 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:22:40.956351 26417 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:39891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:46465
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:46465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.72.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26417
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:40.957315 26417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:40.958281 26417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:40.967506 26427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:40.967608 26417 server_base.cc:1034] running on GCE node
W20260808 18:22:40.967552 26423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:40.967516 26424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:40.968425 26417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:40.969551 26417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:40.970744 26417 hybrid_clock.cc:648] HybridClock initialized: now 1786213360970738 us; error 63 us; skew 500 ppm
Aug 08 18:22:40 dist-test-slave-5k54 krb5kdc[26402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213360, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:40.973739 26417 init.cc:377] Logged in from keytab as kudu/127.23.72.190@KRBTEST.COM (short username kudu)
I20260808 18:22:40.974673 26417 webserver.cc:460] Webserver started at http://127.23.72.190:43935/ using document root <none> and password file <none>
I20260808 18:22:40.975224 26417 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:40.975299 26417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:40.975570 26417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:40.977586 26417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "13670cd371ad403daee382139f48e9c6"
format_stamp: "Formatted at 2026-08-08 18:22:40 on dist-test-slave-5k54"
server_key: "6a2f748d1c86de8effd1775c7a872808"
server_key_iv: "dacae4b2fb6e87299ae93091428c80dc"
server_key_version: "encryptionkey@0"
I20260808 18:22:40.978159 26417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "13670cd371ad403daee382139f48e9c6"
format_stamp: "Formatted at 2026-08-08 18:22:40 on dist-test-slave-5k54"
server_key: "6a2f748d1c86de8effd1775c7a872808"
server_key_iv: "dacae4b2fb6e87299ae93091428c80dc"
server_key_version: "encryptionkey@0"
I20260808 18:22:40.982007 26417 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 18:22:40.984529 26434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:40.985803 26417 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:22:40.985944 26417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "13670cd371ad403daee382139f48e9c6"
format_stamp: "Formatted at 2026-08-08 18:22:40 on dist-test-slave-5k54"
server_key: "6a2f748d1c86de8effd1775c7a872808"
server_key_iv: "dacae4b2fb6e87299ae93091428c80dc"
server_key_version: "encryptionkey@0"
I20260808 18:22:40.986053 26417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:40.996811 26417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:40.999956 26417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:41.000164 26417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:41.007231 26417 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.190:46465
I20260808 18:22:41.007259 26490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.190:46465 every 8 connection(s)
I20260808 18:22:41.008149 26417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:22:41.010721 26491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:41.011991 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 26417
I20260808 18:22:41.012118 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:22:41.012392 23842 external_mini_cluster.cc:1428] Setting key 40055ea736acf4a4d5fb5d7650ad0222
I20260808 18:22:41.016723 26491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13670cd371ad403daee382139f48e9c6: Bootstrap starting.
I20260808 18:22:41.018841 26491 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 13670cd371ad403daee382139f48e9c6: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:41.019613 26491 log.cc:824] T 00000000000000000000000000000000 P 13670cd371ad403daee382139f48e9c6: Log is configured to *not* fsync() on all Append() calls
Aug 08 18:22:41 dist-test-slave-5k54 krb5kdc[26402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213360, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:22:41.021760 26491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13670cd371ad403daee382139f48e9c6: No bootstrap required, opened a new log
I20260808 18:22:41.024909 26491 raft_consensus.cc:348] T 00000000000000000000000000000000 P 13670cd371ad403daee382139f48e9c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13670cd371ad403daee382139f48e9c6" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 46465 } }
I20260808 18:22:41.025110 26491 raft_consensus.cc:374] T 00000000000000000000000000000000 P 13670cd371ad403daee382139f48e9c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:41.025251 26491 raft_consensus.cc:729] T 00000000000000000000000000000000 P 13670cd371ad403daee382139f48e9c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13670cd371ad403daee382139f48e9c6, State: Initialized, Role: FOLLOWER
I20260808 18:22:41.025696 26491 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13670cd371ad403daee382139f48e9c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13670cd371ad403daee382139f48e9c6" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 46465 } }
I20260808 18:22:41.025851 26491 raft_consensus.cc:388] T 00000000000000000000000000000000 P 13670cd371ad403daee382139f48e9c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:22:41.025952 26491 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13670cd371ad403daee382139f48e9c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:22:41.026077 26491 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13670cd371ad403daee382139f48e9c6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:41.027025 26491 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13670cd371ad403daee382139f48e9c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13670cd371ad403daee382139f48e9c6" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 46465 } }
I20260808 18:22:41.027278 26495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:41.013814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42664 (local address 127.23.72.190:46465)
0808 18:22:41.014305 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:41.014324 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:41.014361 (+ 37us) server_negotiation.cc:407] Connection header received
0808 18:22:41.014981 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:41.014997 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:41.015309 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:41.015693 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:41.016514 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:41.017530 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:41.018384 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:41.018849 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:41.019036 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:41.022060 (+ 3024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:41.022073 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:41.022085 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:41.022111 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:41.024342 (+ 2231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:41.024797 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:41.024805 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:41.024814 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:41.024942 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:41.025214 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:41.025221 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:41.025223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:41.025703 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:41.025964 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:41.026190 (+ 226us) server_negotiation.cc:299] Negotiation successful
0808 18:22:41.026481 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":140,"threads_started":1}
I20260808 18:22:41.027408 26491 leader_election.cc:302] T 00000000000000000000000000000000 P 13670cd371ad403daee382139f48e9c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 13670cd371ad403daee382139f48e9c6; no voters:
I20260808 18:22:41.027758 26491 leader_election.cc:288] T 00000000000000000000000000000000 P 13670cd371ad403daee382139f48e9c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:22:41.027866 26497 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 13670cd371ad403daee382139f48e9c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:22:41.028088 26497 raft_consensus.cc:686] T 00000000000000000000000000000000 P 13670cd371ad403daee382139f48e9c6 [term 1 LEADER]: Becoming Leader. State: Replica: 13670cd371ad403daee382139f48e9c6, State: Running, Role: LEADER
I20260808 18:22:41.028447 26497 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13670cd371ad403daee382139f48e9c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13670cd371ad403daee382139f48e9c6" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 46465 } }
I20260808 18:22:41.029012 26491 sys_catalog.cc:564] T 00000000000000000000000000000000 P 13670cd371ad403daee382139f48e9c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:22:41.029521 26497 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13670cd371ad403daee382139f48e9c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13670cd371ad403daee382139f48e9c6. Latest consensus state: current_term: 1 leader_uuid: "13670cd371ad403daee382139f48e9c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13670cd371ad403daee382139f48e9c6" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 46465 } } }
I20260808 18:22:41.029713 26497 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13670cd371ad403daee382139f48e9c6 [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:41.030143 26503 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:22:41.030453 26498 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13670cd371ad403daee382139f48e9c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "13670cd371ad403daee382139f48e9c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13670cd371ad403daee382139f48e9c6" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 46465 } } }
I20260808 18:22:41.030627 26498 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13670cd371ad403daee382139f48e9c6 [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:41.033690 26503 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:22:41.039083 26503 catalog_manager.cc:1347] Generated new cluster ID: 19f24cb04cbb4132b3d25dc5746c896a
I20260808 18:22:41.039167 26503 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:22:41.057013 26503 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:22:41.057948 26503 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:22:41.066419 26503 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 13670cd371ad403daee382139f48e9c6: Generated new TSK 0
I20260808 18:22:41.067085 26503 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.72.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab.
Entry for principal kudu/127.23.72.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab.
I20260808 18:22:41.102788 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.129:0
--local_ip_for_outbound_sockets=127.23.72.129
--webserver_interface=127.23.72.129
--webserver_port=0
--tserver_master_addrs=127.23.72.190:46465
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab
--principal=kudu/127.23.72.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:39891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:41.220021 26520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:41.220247 26520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:41.220286 26520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:41.224108 26520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:41.224152 26520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:41.224205 26520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.129
I20260808 18:22:41.229202 26520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:39891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab
--principal=kudu/127.23.72.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.72.129
--webserver_port=0
--tserver_master_addrs=127.23.72.190:46465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.129
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:41.230089 26520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:41.231069 26520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:41.240187 26528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:41.240204 26526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:41.240204 26525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:41.240523 26520 server_base.cc:1034] running on GCE node
I20260808 18:22:41.240903 26520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:41.241544 26520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:41.242703 26520 hybrid_clock.cc:648] HybridClock initialized: now 1786213361242686 us; error 29 us; skew 500 ppm
Aug 08 18:22:41 dist-test-slave-5k54 krb5kdc[26402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213361, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:41.245569 26520 init.cc:377] Logged in from keytab as kudu/127.23.72.129@KRBTEST.COM (short username kudu)
I20260808 18:22:41.246405 26520 webserver.cc:460] Webserver started at http://127.23.72.129:37377/ using document root <none> and password file <none>
I20260808 18:22:41.246949 26520 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:41.247009 26520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:41.247244 26520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:41.249365 26520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-0/data/instance:
uuid: "a6b1707c1c2646838e2a6d64685d68cd"
format_stamp: "Formatted at 2026-08-08 18:22:41 on dist-test-slave-5k54"
server_key: "2ca4e988496c9a78c48e32f5f3309730"
server_key_iv: "862f0fbb698403662447dbfb66cf2f59"
server_key_version: "encryptionkey@0"
I20260808 18:22:41.249943 26520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance:
uuid: "a6b1707c1c2646838e2a6d64685d68cd"
format_stamp: "Formatted at 2026-08-08 18:22:41 on dist-test-slave-5k54"
server_key: "2ca4e988496c9a78c48e32f5f3309730"
server_key_iv: "862f0fbb698403662447dbfb66cf2f59"
server_key_version: "encryptionkey@0"
I20260808 18:22:41.253786 26520 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 18:22:41.256232 26535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:41.257444 26520 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:22:41.257587 26520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-0/wal
uuid: "a6b1707c1c2646838e2a6d64685d68cd"
format_stamp: "Formatted at 2026-08-08 18:22:41 on dist-test-slave-5k54"
server_key: "2ca4e988496c9a78c48e32f5f3309730"
server_key_iv: "862f0fbb698403662447dbfb66cf2f59"
server_key_version: "encryptionkey@0"
I20260808 18:22:41.257704 26520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:41.272584 26520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:41.275365 26520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:41.275583 26520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:41.276165 26520 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:22:41.276965 26520 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:22:41.277037 26520 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:41.277109 26520 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:22:41.277163 26520 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:41.285979 26520 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.129:44001
I20260808 18:22:41.286007 26655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.129:44001 every 8 connection(s)
I20260808 18:22:41.286866 26520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
I20260808 18:22:41.289989 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 26520
I20260808 18:22:41.290144 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance
I20260808 18:22:41.290418 23842 external_mini_cluster.cc:1428] Setting key 068ec3a26346b052eea418dfd91abd1a
Aug 08 18:22:41 dist-test-slave-5k54 krb5kdc[26402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213361, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.72.130@KRBTEST.COM; defaulting to no policy
I20260808 18:22:41.300616 26495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:41.288693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:58711 (local address 127.23.72.190:46465)
0808 18:22:41.288843 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:41.288848 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:41.289775 (+ 927us) server_negotiation.cc:407] Connection header received
0808 18:22:41.291268 (+ 1493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:41.291271 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:41.291325 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:41.291426 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:41.292662 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:41.293205 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:41.294329 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:41.294563 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:41.294675 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:41.297078 (+ 2403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:41.297081 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:41.297084 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:41.297115 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:41.298538 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:41.299051 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:41.299055 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:41.299057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:41.299138 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:41.299518 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:41.299522 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:41.299523 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:41.299708 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:41.299814 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:41.300331 (+ 517us) server_negotiation.cc:299] Negotiation successful
0808 18:22:41.300448 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 18:22:41.301366 26658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:41.289177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46465 (local address 127.23.72.129:58711)
0808 18:22:41.289615 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:41.289656 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:41.291071 (+ 1415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:41.291618 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:41.291624 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:41.292071 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:41.292524 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:41.292537 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:41.293340 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:41.293344 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:41.294204 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:41.294213 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:41.294710 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:41.294714 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:41.294877 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:41.295528 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:41.295543 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:41.296923 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:41.298671 (+ 1748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:41.298677 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:41.298690 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:41.298945 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:41.299229 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:41.299233 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:41.299235 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:41.299369 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:41.299813 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:41.299822 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:41.300086 (+ 264us) client_negotiation.cc:769] Sending connection context
0808 18:22:41.300273 (+ 187us) client_negotiation.cc:240] Negotiation successful
0808 18:22:41.300576 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":107,"threads_started":1}
I20260808 18:22:41.302520 26656 heartbeater.cc:347] Connected to a master server at 127.23.72.190:46465
I20260808 18:22:41.302821 26656 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.130@KRBTEST.COM" created.
I20260808 18:22:41.303444 26656 heartbeater.cc:511] Master 127.23.72.190:46465 requested a full tablet report, sending...
I20260808 18:22:41.305161 26452 ts_manager.cc:194] Registered new tserver with Master: a6b1707c1c2646838e2a6d64685d68cd (127.23.72.129:44001)
I20260808 18:22:41.306851 26452 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.129@KRBTEST.COM'} at 127.23.72.129:58711
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab.
Entry for principal kudu/127.23.72.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab.
I20260808 18:22:41.320497 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.130:0
--local_ip_for_outbound_sockets=127.23.72.130
--webserver_interface=127.23.72.130
--webserver_port=0
--tserver_master_addrs=127.23.72.190:46465
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab
--principal=kudu/127.23.72.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:39891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:41.430594 26661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:41.430919 26661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:41.430966 26661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:41.434963 26661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:41.435025 26661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:41.435082 26661 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.130
I20260808 18:22:41.440222 26661 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:39891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab
--principal=kudu/127.23.72.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.72.130
--webserver_port=0
--tserver_master_addrs=127.23.72.190:46465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.130
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:41.441167 26661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:41.442091 26661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:41.451258 26666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:41.451274 26669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:41.451263 26667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:41.451485 26661 server_base.cc:1034] running on GCE node
I20260808 18:22:41.451889 26661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:41.452499 26661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:41.453698 26661 hybrid_clock.cc:648] HybridClock initialized: now 1786213361453677 us; error 29 us; skew 500 ppm
Aug 08 18:22:41 dist-test-slave-5k54 krb5kdc[26402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213361, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:41.456734 26661 init.cc:377] Logged in from keytab as kudu/127.23.72.130@KRBTEST.COM (short username kudu)
I20260808 18:22:41.457613 26661 webserver.cc:460] Webserver started at http://127.23.72.130:32921/ using document root <none> and password file <none>
I20260808 18:22:41.458160 26661 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:41.458222 26661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:41.458454 26661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:41.460517 26661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-1/data/instance:
uuid: "75e84905bfea4cd9a12fe85e0d52367f"
format_stamp: "Formatted at 2026-08-08 18:22:41 on dist-test-slave-5k54"
server_key: "5cf545c7a164f430b633394e86c8b3ef"
server_key_iv: "fecc3224176c47a41cb19e4e1022f7d3"
server_key_version: "encryptionkey@0"
I20260808 18:22:41.461107 26661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance:
uuid: "75e84905bfea4cd9a12fe85e0d52367f"
format_stamp: "Formatted at 2026-08-08 18:22:41 on dist-test-slave-5k54"
server_key: "5cf545c7a164f430b633394e86c8b3ef"
server_key_iv: "fecc3224176c47a41cb19e4e1022f7d3"
server_key_version: "encryptionkey@0"
I20260808 18:22:41.464838 26661 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:22:41.467196 26676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:41.468448 26661 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:22:41.468568 26661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-1/wal
uuid: "75e84905bfea4cd9a12fe85e0d52367f"
format_stamp: "Formatted at 2026-08-08 18:22:41 on dist-test-slave-5k54"
server_key: "5cf545c7a164f430b633394e86c8b3ef"
server_key_iv: "fecc3224176c47a41cb19e4e1022f7d3"
server_key_version: "encryptionkey@0"
I20260808 18:22:41.468668 26661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:41.481369 26661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:41.484098 26661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:41.484298 26661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:41.484869 26661 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:22:41.485630 26661 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:22:41.485677 26661 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:41.485743 26661 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:22:41.485788 26661 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:41.494943 26661 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.130:45961
I20260808 18:22:41.494967 26799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.130:45961 every 8 connection(s)
I20260808 18:22:41.495929 26661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
I20260808 18:22:41.496800 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 26661
I20260808 18:22:41.496933 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance
I20260808 18:22:41.497232 23842 external_mini_cluster.cc:1428] Setting key 76df6fed8b4ede1a9c191364ace299c5
Aug 08 18:22:41 dist-test-slave-5k54 krb5kdc[26402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213361, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.130@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.72.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.131@KRBTEST.COM" created.
I20260808 18:22:41.512183 26495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:41.498855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:56025 (local address 127.23.72.190:46465)
0808 18:22:41.499015 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:41.499019 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:41.500053 (+ 1034us) server_negotiation.cc:407] Connection header received
0808 18:22:41.501631 (+ 1578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:41.501637 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:41.501698 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:41.501798 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:41.502884 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:41.503815 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:41.505028 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:41.505468 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:41.505600 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:41.508389 (+ 2789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:41.508394 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:41.508399 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:41.508433 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:41.510065 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:41.510598 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:41.510602 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:41.510603 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:41.510685 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:41.511017 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:41.511021 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:41.511023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:41.511203 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:41.511308 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:41.511944 (+ 636us) server_negotiation.cc:299] Negotiation successful
0808 18:22:41.512055 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 18:22:41.513306 26802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:41.499284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46465 (local address 127.23.72.130:56025)
0808 18:22:41.499874 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:41.499933 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:41.501420 (+ 1487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:41.501924 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:41.501931 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:41.502269 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:41.502732 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:41.502744 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:41.503958 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:41.503963 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:41.504891 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:41.504901 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:41.505619 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:41.505622 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:41.505764 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:41.506399 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:41.506414 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:41.508199 (+ 1785us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:41.510205 (+ 2006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:41.510210 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:41.510231 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:41.510485 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:41.510777 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:41.510781 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:41.510783 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:41.510919 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:41.511320 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:41.511328 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:41.511647 (+ 319us) client_negotiation.cc:769] Sending connection context
0808 18:22:41.511843 (+ 196us) client_negotiation.cc:240] Negotiation successful
0808 18:22:41.512202 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":164,"threads_started":1}
I20260808 18:22:41.514495 26800 heartbeater.cc:347] Connected to a master server at 127.23.72.190:46465
I20260808 18:22:41.514747 26800 heartbeater.cc:464] Registering TS with master...
I20260808 18:22:41.515231 26800 heartbeater.cc:511] Master 127.23.72.190:46465 requested a full tablet report, sending...
I20260808 18:22:41.516281 26452 ts_manager.cc:194] Registered new tserver with Master: 75e84905bfea4cd9a12fe85e0d52367f (127.23.72.130:45961)
I20260808 18:22:41.516937 26452 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.130@KRBTEST.COM'} at 127.23.72.130:56025
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab.
Entry for principal kudu/127.23.72.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab.
I20260808 18:22:41.530835 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.131:0
--local_ip_for_outbound_sockets=127.23.72.131
--webserver_interface=127.23.72.131
--webserver_port=0
--tserver_master_addrs=127.23.72.190:46465
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab
--principal=kudu/127.23.72.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:39891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:41.640565 26805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:41.640798 26805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:41.640841 26805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:41.644742 26805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:41.644805 26805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:41.644865 26805 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.131
I20260808 18:22:41.650151 26805 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:39891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab
--principal=kudu/127.23.72.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.72.131
--webserver_port=0
--tserver_master_addrs=127.23.72.190:46465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26805
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.131
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:41.651086 26805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:41.652076 26805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:41.661399 26813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:41.661480 26805 server_base.cc:1034] running on GCE node
W20260808 18:22:41.661432 26810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:41.661427 26811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:41.662194 26805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:41.662776 26805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:41.663959 26805 hybrid_clock.cc:648] HybridClock initialized: now 1786213361663941 us; error 29 us; skew 500 ppm
Aug 08 18:22:41 dist-test-slave-5k54 krb5kdc[26402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213361, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:41.666734 26805 init.cc:377] Logged in from keytab as kudu/127.23.72.131@KRBTEST.COM (short username kudu)
I20260808 18:22:41.667641 26805 webserver.cc:460] Webserver started at http://127.23.72.131:45959/ using document root <none> and password file <none>
I20260808 18:22:41.668167 26805 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:41.668255 26805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:41.668483 26805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:41.670424 26805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-2/data/instance:
uuid: "d3806a0606604de18e8342db8e661db7"
format_stamp: "Formatted at 2026-08-08 18:22:41 on dist-test-slave-5k54"
server_key: "092c7e1426d5ae00a597cb7684470c91"
server_key_iv: "3ef874ca7eb8c99f01c3b8479d494bbb"
server_key_version: "encryptionkey@0"
I20260808 18:22:41.670974 26805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance:
uuid: "d3806a0606604de18e8342db8e661db7"
format_stamp: "Formatted at 2026-08-08 18:22:41 on dist-test-slave-5k54"
server_key: "092c7e1426d5ae00a597cb7684470c91"
server_key_iv: "3ef874ca7eb8c99f01c3b8479d494bbb"
server_key_version: "encryptionkey@0"
I20260808 18:22:41.674711 26805 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260808 18:22:41.677120 26822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:41.678280 26805 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:22:41.678419 26805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-2/wal
uuid: "d3806a0606604de18e8342db8e661db7"
format_stamp: "Formatted at 2026-08-08 18:22:41 on dist-test-slave-5k54"
server_key: "092c7e1426d5ae00a597cb7684470c91"
server_key_iv: "3ef874ca7eb8c99f01c3b8479d494bbb"
server_key_version: "encryptionkey@0"
I20260808 18:22:41.678525 26805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:41.702843 26805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:41.706480 26805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:41.706702 26805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:41.707445 26805 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:22:41.708374 26805 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:22:41.708462 26805 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:41.708539 26805 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:22:41.708597 26805 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:41.718734 26805 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.131:46793
I20260808 18:22:41.718751 26947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.131:46793 every 8 connection(s)
I20260808 18:22:41.719758 26805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
I20260808 18:22:41.727631 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 26805
I20260808 18:22:41.727826 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance
I20260808 18:22:41.728226 23842 external_mini_cluster.cc:1428] Setting key 2306543e0cff842a8fbde15cae6d26bb
Aug 08 18:22:41 dist-test-slave-5k54 krb5kdc[26402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213361, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.131@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:22:41.734067 26495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:41.721683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:33623 (local address 127.23.72.190:46465)
0808 18:22:41.721837 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:41.721840 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:41.722764 (+ 924us) server_negotiation.cc:407] Connection header received
0808 18:22:41.724066 (+ 1302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:41.724069 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:41.724120 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:41.724202 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:41.725344 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:41.725919 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:41.727046 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:41.727339 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:41.727513 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:41.730463 (+ 2950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:41.730466 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:41.730468 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:41.730492 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:41.732248 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:41.732724 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:41.732726 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:41.732728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:41.732809 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:41.733102 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:41.733104 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:41.733106 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:41.733298 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:41.733381 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:41.733843 (+ 462us) server_negotiation.cc:299] Negotiation successful
0808 18:22:41.733945 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 18:22:41.734848 26951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:41.722107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46465 (local address 127.23.72.131:33623)
0808 18:22:41.722575 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:41.722617 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:41.723857 (+ 1240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:41.724372 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:41.724378 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:41.724720 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:41.725197 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:41.725210 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:41.726043 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:41.726046 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:41.726924 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:41.726933 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:41.727547 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:41.727550 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:41.727702 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:41.728504 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:41.728530 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:41.730305 (+ 1775us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:41.732367 (+ 2062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:41.732371 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:41.732382 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:41.732633 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:41.732891 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:41.732893 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:41.732894 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:41.733020 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:41.733393 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:41.733397 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:41.733632 (+ 235us) client_negotiation.cc:769] Sending connection context
0808 18:22:41.733779 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 18:22:41.734011 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":136,"threads_started":1}
I20260808 18:22:41.736089 26948 heartbeater.cc:347] Connected to a master server at 127.23.72.190:46465
I20260808 18:22:41.736313 26948 heartbeater.cc:464] Registering TS with master...
I20260808 18:22:41.736773 26948 heartbeater.cc:511] Master 127.23.72.190:46465 requested a full tablet report, sending...
I20260808 18:22:41.737766 26452 ts_manager.cc:194] Registered new tserver with Master: d3806a0606604de18e8342db8e661db7 (127.23.72.131:46793)
I20260808 18:22:41.738320 26452 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.131@KRBTEST.COM'} at 127.23.72.131:33623
I20260808 18:22:41.742065 23842 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 18:22:41.756433 26495 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:22:41.754954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42672 (local address 127.23.72.190:46465)
0808 18:22:41.755110 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:41.755115 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:41.755264 (+ 149us) server_negotiation.cc:407] Connection header received
0808 18:22:41.755444 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:41.755447 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:41.755494 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:41.755577 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:41.756278 (+ 701us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42672: BlockingRecv error: recv got EOF from 127.0.0.1:42672 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 18:22:41.757572 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 26520
I20260808 18:22:41.764928 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 26661
I20260808 18:22:41.772462 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 26805
I20260808 18:22:41.780921 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 26417
2026-08-08T18:22:41Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2159 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213345431840-23842-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:22:41 dist-test-slave-5k54 krb5kdc[26962](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:22:41 dist-test-slave-5k54 krb5kdc[26962](info): set up 2 sockets
Aug 08 18:22:41 dist-test-slave-5k54 krb5kdc[26962](info): commencing operation
krb5kdc: starting...
W20260808 18:22:42.868573 23842 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Aug 08 18:22:42 dist-test-slave-5k54 krb5kdc[26962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213362, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:22:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:22:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.72.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
I20260808 18:22:42.996402 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:40995
--webserver_interface=127.23.72.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:45273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:40995
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:43.106819 26977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:43.107085 26977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:43.107126 26977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:43.111272 26977 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:22:43.111334 26977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:43.111353 26977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:43.111370 26977 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:22:43.111385 26977 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:22:43.116677 26977 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:45273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:40995
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:40995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.72.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26977
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:43.117760 26977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:43.118816 26977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:43.127374 26983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:43.127374 26986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:43.127485 26984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:43.127638 26977 server_base.cc:1034] running on GCE node
I20260808 18:22:43.128254 26977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:43.129442 26977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:43.130666 26977 hybrid_clock.cc:648] HybridClock initialized: now 1786213363130652 us; error 35 us; skew 500 ppm
Aug 08 18:22:43 dist-test-slave-5k54 krb5kdc[26962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213363, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:43.133634 26977 init.cc:377] Logged in from keytab as kudu/127.23.72.190@KRBTEST.COM (short username kudu)
I20260808 18:22:43.134482 26977 webserver.cc:460] Webserver started at http://127.23.72.190:42297/ using document root <none> and password file <none>
I20260808 18:22:43.135073 26977 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:43.135170 26977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:43.135401 26977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:43.137416 26977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "5328f82c1ad645059734b283d30ba048"
format_stamp: "Formatted at 2026-08-08 18:22:43 on dist-test-slave-5k54"
server_key: "a9ed538747c8ef5b0e6da0eab3ef2742"
server_key_iv: "09ff30eb235f52ef35ed86f2a827f2d7"
server_key_version: "encryptionkey@0"
I20260808 18:22:43.138013 26977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "5328f82c1ad645059734b283d30ba048"
format_stamp: "Formatted at 2026-08-08 18:22:43 on dist-test-slave-5k54"
server_key: "a9ed538747c8ef5b0e6da0eab3ef2742"
server_key_iv: "09ff30eb235f52ef35ed86f2a827f2d7"
server_key_version: "encryptionkey@0"
I20260808 18:22:43.141625 26977 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260808 18:22:43.143919 26993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:43.145134 26977 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:22:43.145272 26977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "5328f82c1ad645059734b283d30ba048"
format_stamp: "Formatted at 2026-08-08 18:22:43 on dist-test-slave-5k54"
server_key: "a9ed538747c8ef5b0e6da0eab3ef2742"
server_key_iv: "09ff30eb235f52ef35ed86f2a827f2d7"
server_key_version: "encryptionkey@0"
I20260808 18:22:43.145378 26977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:43.160440 26977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:43.163722 26977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:43.163930 26977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:43.170861 26977 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.190:40995
I20260808 18:22:43.171176 27050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.190:40995 every 8 connection(s)
I20260808 18:22:43.171813 26977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:22:43.172590 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 26977
I20260808 18:22:43.172724 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:22:43.173094 23842 external_mini_cluster.cc:1428] Setting key 83c779ad6de2c57124478ac099c50d68
I20260808 18:22:43.175283 27051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:43.181610 27051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5328f82c1ad645059734b283d30ba048: Bootstrap starting.
Aug 08 18:22:43 dist-test-slave-5k54 krb5kdc[26962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213362, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:22:43.184258 27051 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5328f82c1ad645059734b283d30ba048: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:43.185186 27051 log.cc:824] T 00000000000000000000000000000000 P 5328f82c1ad645059734b283d30ba048: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:43.187547 27051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5328f82c1ad645059734b283d30ba048: No bootstrap required, opened a new log
I20260808 18:22:43.188470 27054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:43.174473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36664 (local address 127.23.72.190:40995)
0808 18:22:43.174920 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:43.174934 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:43.174968 (+ 34us) server_negotiation.cc:407] Connection header received
0808 18:22:43.175707 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:43.175724 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:43.176039 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:43.176361 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:43.177252 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:43.178747 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:43.179667 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:43.180361 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:43.180536 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:43.183353 (+ 2817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:43.183357 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:43.183370 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:43.183395 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:43.185417 (+ 2022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:43.185927 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:43.185931 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:43.185935 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:43.186090 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:43.186360 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:43.186363 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:43.186364 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:43.186852 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:43.187038 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:43.187241 (+ 203us) server_negotiation.cc:299] Negotiation successful
0808 18:22:43.187548 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":151,"threads_started":1}
I20260808 18:22:43.191349 23842 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:22:43.191874 27051 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5328f82c1ad645059734b283d30ba048 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5328f82c1ad645059734b283d30ba048" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40995 } }
I20260808 18:22:43.192082 27051 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5328f82c1ad645059734b283d30ba048 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:43.192157 27051 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5328f82c1ad645059734b283d30ba048 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5328f82c1ad645059734b283d30ba048, State: Initialized, Role: FOLLOWER
I20260808 18:22:43.192559 27051 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5328f82c1ad645059734b283d30ba048 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5328f82c1ad645059734b283d30ba048" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40995 } }
I20260808 18:22:43.192674 27051 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5328f82c1ad645059734b283d30ba048 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:22:43.192756 27051 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5328f82c1ad645059734b283d30ba048 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:22:43.192857 27051 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5328f82c1ad645059734b283d30ba048 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:43.193732 27051 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5328f82c1ad645059734b283d30ba048 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5328f82c1ad645059734b283d30ba048" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40995 } }
I20260808 18:22:43.194072 27051 leader_election.cc:302] T 00000000000000000000000000000000 P 5328f82c1ad645059734b283d30ba048 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5328f82c1ad645059734b283d30ba048; no voters:
I20260808 18:22:43.194394 27051 leader_election.cc:288] T 00000000000000000000000000000000 P 5328f82c1ad645059734b283d30ba048 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:22:43.194533 27057 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5328f82c1ad645059734b283d30ba048 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:22:43.194802 27057 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5328f82c1ad645059734b283d30ba048 [term 1 LEADER]: Becoming Leader. State: Replica: 5328f82c1ad645059734b283d30ba048, State: Running, Role: LEADER
I20260808 18:22:43.195163 27057 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5328f82c1ad645059734b283d30ba048 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5328f82c1ad645059734b283d30ba048" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40995 } }
I20260808 18:22:43.195708 27051 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5328f82c1ad645059734b283d30ba048 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:22:43.196306 27057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5328f82c1ad645059734b283d30ba048 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5328f82c1ad645059734b283d30ba048. Latest consensus state: current_term: 1 leader_uuid: "5328f82c1ad645059734b283d30ba048" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5328f82c1ad645059734b283d30ba048" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40995 } } }
I20260808 18:22:43.196822 27059 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5328f82c1ad645059734b283d30ba048 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5328f82c1ad645059734b283d30ba048" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5328f82c1ad645059734b283d30ba048" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40995 } } }
I20260808 18:22:43.198607 27059 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5328f82c1ad645059734b283d30ba048 [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:43.198596 27057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5328f82c1ad645059734b283d30ba048 [sys.catalog]: This master's current role is: LEADER
W20260808 18:22:43.200997 27074 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5328f82c1ad645059734b283d30ba048: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:22:43.201113 27074 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:22:43.201220 27077 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:22:43.202284 27077 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:22:43.208611 27077 catalog_manager.cc:1347] Generated new cluster ID: b954051711fa4b5ea346370490da9be3
I20260808 18:22:43.208671 27077 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:22:43.214184 27054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:43.207951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36674 (local address 127.23.72.190:40995)
0808 18:22:43.208059 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:43.208065 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:43.208999 (+ 934us) server_negotiation.cc:407] Connection header received
0808 18:22:43.209140 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:43.209143 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:43.209197 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:43.209292 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:43.209880 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:43.210691 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:43.211849 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:43.212218 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:43.212335 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:43.213202 (+ 867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:43.213206 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:43.213209 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:22:43.213705 (+ 496us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:43.213739 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:22:43.213758 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:43.213858 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:43.213957 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 18:22:43.214050 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 18:22:43.220261 27077 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:22:43.221365 27077 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:22:43.234488 27077 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5328f82c1ad645059734b283d30ba048: Generated new TSK 0
I20260808 18:22:43.235317 27077 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:22:43.239923 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 26977
2026-08-08T18:22:43Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1457 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:22:43 dist-test-slave-5k54 krb5kdc[27088](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:22:43 dist-test-slave-5k54 krb5kdc[27088](info): set up 2 sockets
Aug 08 18:22:43 dist-test-slave-5k54 krb5kdc[27088](info): commencing operation
krb5kdc: starting...
W20260808 18:22:44.332363 23842 mini_kdc.cc:121] Time spent starting KDC: real 1.070s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Aug 08 18:22:44 dist-test-slave-5k54 krb5kdc[27088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213364, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:22:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:22:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.72.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
I20260808 18:22:44.459146 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:38711
--webserver_interface=127.23.72.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:42585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:38711
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:44.570637 27102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:44.571100 27102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:44.571228 27102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:44.575440 27102 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:22:44.575536 27102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:44.575574 27102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:44.575611 27102 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:22:44.575645 27102 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:22:44.581070 27102 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:42585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:38711
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:38711
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.72.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27102
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:44.582261 27102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:44.583323 27102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:44.592206 27109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:44.592200 27108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:44.592285 27102 server_base.cc:1034] running on GCE node
W20260808 18:22:44.592206 27112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:44.593076 27102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:44.594092 27102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:44.595252 27102 hybrid_clock.cc:648] HybridClock initialized: now 1786213364595233 us; error 31 us; skew 500 ppm
Aug 08 18:22:44 dist-test-slave-5k54 krb5kdc[27088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213364, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:44.598174 27102 init.cc:377] Logged in from keytab as kudu/127.23.72.190@KRBTEST.COM (short username kudu)
I20260808 18:22:44.599085 27102 webserver.cc:460] Webserver started at http://127.23.72.190:35553/ using document root <none> and password file <none>
I20260808 18:22:44.599699 27102 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:44.599763 27102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:44.600003 27102 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:44.602013 27102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "ffd32d919813495bb0541cdc0cd88d21"
format_stamp: "Formatted at 2026-08-08 18:22:44 on dist-test-slave-5k54"
server_key: "785b58b4128c101fe0b04818673b35f1"
server_key_iv: "9cee8cd4408a6671d8323273203848a3"
server_key_version: "encryptionkey@0"
I20260808 18:22:44.602582 27102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "ffd32d919813495bb0541cdc0cd88d21"
format_stamp: "Formatted at 2026-08-08 18:22:44 on dist-test-slave-5k54"
server_key: "785b58b4128c101fe0b04818673b35f1"
server_key_iv: "9cee8cd4408a6671d8323273203848a3"
server_key_version: "encryptionkey@0"
I20260808 18:22:44.606441 27102 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 18:22:44.608917 27119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:44.610150 27102 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:22:44.610291 27102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "ffd32d919813495bb0541cdc0cd88d21"
format_stamp: "Formatted at 2026-08-08 18:22:44 on dist-test-slave-5k54"
server_key: "785b58b4128c101fe0b04818673b35f1"
server_key_iv: "9cee8cd4408a6671d8323273203848a3"
server_key_version: "encryptionkey@0"
I20260808 18:22:44.610415 27102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:44.628716 27102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:44.632176 27102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:44.632393 27102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:44.639869 27102 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.190:38711
I20260808 18:22:44.640177 27174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.190:38711 every 8 connection(s)
I20260808 18:22:44.640858 27102 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:22:44.643874 27175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:44.646303 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 27102
I20260808 18:22:44.646443 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:22:44.646781 23842 external_mini_cluster.cc:1428] Setting key 5271729e38a63a35ca9a62324d111fdb
I20260808 18:22:44.650750 27175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffd32d919813495bb0541cdc0cd88d21: Bootstrap starting.
I20260808 18:22:44.653294 27175 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ffd32d919813495bb0541cdc0cd88d21: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:44.654114 27175 log.cc:824] T 00000000000000000000000000000000 P ffd32d919813495bb0541cdc0cd88d21: Log is configured to *not* fsync() on all Append() calls
Aug 08 18:22:44 dist-test-slave-5k54 krb5kdc[27088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213364, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:22:44.656589 27175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffd32d919813495bb0541cdc0cd88d21: No bootstrap required, opened a new log
I20260808 18:22:44.660400 27175 raft_consensus.cc:348] T 00000000000000000000000000000000 P ffd32d919813495bb0541cdc0cd88d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffd32d919813495bb0541cdc0cd88d21" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 38711 } }
I20260808 18:22:44.660717 27175 raft_consensus.cc:374] T 00000000000000000000000000000000 P ffd32d919813495bb0541cdc0cd88d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:44.660810 27175 raft_consensus.cc:729] T 00000000000000000000000000000000 P ffd32d919813495bb0541cdc0cd88d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffd32d919813495bb0541cdc0cd88d21, State: Initialized, Role: FOLLOWER
I20260808 18:22:44.661334 27175 consensus_queue.cc:260] T 00000000000000000000000000000000 P ffd32d919813495bb0541cdc0cd88d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffd32d919813495bb0541cdc0cd88d21" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 38711 } }
I20260808 18:22:44.661475 27175 raft_consensus.cc:388] T 00000000000000000000000000000000 P ffd32d919813495bb0541cdc0cd88d21 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:22:44.661571 27175 raft_consensus.cc:482] T 00000000000000000000000000000000 P ffd32d919813495bb0541cdc0cd88d21 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:22:44.661702 27175 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ffd32d919813495bb0541cdc0cd88d21 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:44.661870 27178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:44.648376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42456 (local address 127.23.72.190:38711)
0808 18:22:44.648849 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:44.648865 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:44.648908 (+ 43us) server_negotiation.cc:407] Connection header received
0808 18:22:44.649750 (+ 842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:44.649770 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:44.650128 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:44.650476 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:44.651326 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:44.652384 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:44.653259 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:44.653793 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:44.653971 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:44.656952 (+ 2981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:44.656957 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:44.656970 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:44.656997 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:44.658739 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:44.659201 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:44.659206 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:44.659211 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:44.659323 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:44.659639 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:44.659643 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:44.659644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:44.660042 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:44.660259 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:44.660510 (+ 251us) server_negotiation.cc:299] Negotiation successful
0808 18:22:44.660918 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":148,"threads_started":1}
I20260808 18:22:44.662778 27175 raft_consensus.cc:504] T 00000000000000000000000000000000 P ffd32d919813495bb0541cdc0cd88d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffd32d919813495bb0541cdc0cd88d21" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 38711 } }
I20260808 18:22:44.663192 27175 leader_election.cc:302] T 00000000000000000000000000000000 P ffd32d919813495bb0541cdc0cd88d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ffd32d919813495bb0541cdc0cd88d21; no voters:
I20260808 18:22:44.663614 27175 leader_election.cc:288] T 00000000000000000000000000000000 P ffd32d919813495bb0541cdc0cd88d21 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:22:44.663731 27180 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ffd32d919813495bb0541cdc0cd88d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:22:44.663969 27180 raft_consensus.cc:686] T 00000000000000000000000000000000 P ffd32d919813495bb0541cdc0cd88d21 [term 1 LEADER]: Becoming Leader. State: Replica: ffd32d919813495bb0541cdc0cd88d21, State: Running, Role: LEADER
I20260808 18:22:44.664320 27180 consensus_queue.cc:237] T 00000000000000000000000000000000 P ffd32d919813495bb0541cdc0cd88d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffd32d919813495bb0541cdc0cd88d21" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 38711 } }
I20260808 18:22:44.664944 27175 sys_catalog.cc:564] T 00000000000000000000000000000000 P ffd32d919813495bb0541cdc0cd88d21 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:22:44.665426 27180 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffd32d919813495bb0541cdc0cd88d21 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ffd32d919813495bb0541cdc0cd88d21. Latest consensus state: current_term: 1 leader_uuid: "ffd32d919813495bb0541cdc0cd88d21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffd32d919813495bb0541cdc0cd88d21" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 38711 } } }
I20260808 18:22:44.665613 27180 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffd32d919813495bb0541cdc0cd88d21 [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:44.665874 27181 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffd32d919813495bb0541cdc0cd88d21 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ffd32d919813495bb0541cdc0cd88d21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffd32d919813495bb0541cdc0cd88d21" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 38711 } } }
I20260808 18:22:44.666020 27181 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffd32d919813495bb0541cdc0cd88d21 [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:44.666092 27185 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:22:44.669040 27185 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:22:44.674019 27185 catalog_manager.cc:1347] Generated new cluster ID: 47070138a417404da422dc4cecdf780c
I20260808 18:22:44.674084 27185 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:22:44.706284 27185 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:22:44.707651 27185 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:22:44.714435 27185 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ffd32d919813495bb0541cdc0cd88d21: Generated new TSK 0
I20260808 18:22:44.715322 27185 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.72.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab.
Entry for principal kudu/127.23.72.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab.
I20260808 18:22:44.750289 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.129:0
--local_ip_for_outbound_sockets=127.23.72.129
--webserver_interface=127.23.72.129
--webserver_port=0
--tserver_master_addrs=127.23.72.190:38711
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab
--principal=kudu/127.23.72.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:42585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:44.862313 27201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:44.862550 27201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:44.862591 27201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:44.866672 27201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:44.866736 27201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:44.866822 27201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.129
I20260808 18:22:44.874636 27201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:42585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab
--principal=kudu/127.23.72.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.72.129
--webserver_port=0
--tserver_master_addrs=127.23.72.190:38711
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.129
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:44.875831 27201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:44.877060 27201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:44.887068 27207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:44.887106 27210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:44.887106 27208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:44.887394 27201 server_base.cc:1034] running on GCE node
I20260808 18:22:44.887821 27201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:44.888427 27201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:44.889611 27201 hybrid_clock.cc:648] HybridClock initialized: now 1786213364889600 us; error 33 us; skew 500 ppm
Aug 08 18:22:44 dist-test-slave-5k54 krb5kdc[27088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213364, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:44.897327 27201 init.cc:377] Logged in from keytab as kudu/127.23.72.129@KRBTEST.COM (short username kudu)
I20260808 18:22:44.898203 27201 webserver.cc:460] Webserver started at http://127.23.72.129:39261/ using document root <none> and password file <none>
I20260808 18:22:44.898720 27201 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:44.898778 27201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:44.898957 27201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:44.900990 27201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-0/data/instance:
uuid: "4667296435b348b68b5f1796aef6a38c"
format_stamp: "Formatted at 2026-08-08 18:22:44 on dist-test-slave-5k54"
server_key: "24eedd523c7c56e7217530f53abfcb16"
server_key_iv: "94fd74038140b05c88daef54c64dc315"
server_key_version: "encryptionkey@0"
I20260808 18:22:44.901486 27201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance:
uuid: "4667296435b348b68b5f1796aef6a38c"
format_stamp: "Formatted at 2026-08-08 18:22:44 on dist-test-slave-5k54"
server_key: "24eedd523c7c56e7217530f53abfcb16"
server_key_iv: "94fd74038140b05c88daef54c64dc315"
server_key_version: "encryptionkey@0"
I20260808 18:22:44.905108 27201 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20260808 18:22:44.907590 27217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:44.908799 27201 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 18:22:44.908892 27201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-0/wal
uuid: "4667296435b348b68b5f1796aef6a38c"
format_stamp: "Formatted at 2026-08-08 18:22:44 on dist-test-slave-5k54"
server_key: "24eedd523c7c56e7217530f53abfcb16"
server_key_iv: "94fd74038140b05c88daef54c64dc315"
server_key_version: "encryptionkey@0"
I20260808 18:22:44.908979 27201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:44.920475 27201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:44.923537 27201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:44.923681 27201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:44.924170 27201 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:22:44.924925 27201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:22:44.924973 27201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:44.925012 27201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:22:44.925027 27201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:44.933758 27201 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.129:40971
I20260808 18:22:44.933789 27341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.129:40971 every 8 connection(s)
I20260808 18:22:44.934597 27201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
I20260808 18:22:44.937069 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 27201
I20260808 18:22:44.937211 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance
I20260808 18:22:44.937543 23842 external_mini_cluster.cc:1428] Setting key 0ec4f77816567ccd0b5f1adf1095e13c
Aug 08 18:22:44 dist-test-slave-5k54 krb5kdc[27088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213364, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.72.130@KRBTEST.COM; defaulting to no policy
I20260808 18:22:44.949095 27178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:44.936414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:39181 (local address 127.23.72.190:38711)
0808 18:22:44.936562 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:44.936566 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:44.937510 (+ 944us) server_negotiation.cc:407] Connection header received
0808 18:22:44.939112 (+ 1602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:44.939115 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:44.939170 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:44.939252 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:44.940487 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:44.941040 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:44.942185 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:44.942427 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:44.942541 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:44.945506 (+ 2965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:44.945509 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:44.945511 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:44.945535 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:44.946973 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:44.947610 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:44.947612 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:44.947613 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:44.947688 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:44.948092 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:44.948095 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:44.948096 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:44.948271 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:44.948347 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:44.948852 (+ 505us) server_negotiation.cc:299] Negotiation successful
0808 18:22:44.948972 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 18:22:44.949858 27344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:44.936742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:38711 (local address 127.23.72.129:39181)
0808 18:22:44.937357 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:44.937399 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:44.938865 (+ 1466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:44.939491 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:44.939500 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:44.939864 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:44.940328 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:44.940341 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:44.941180 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:44.941183 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:44.942054 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:44.942063 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:44.942580 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:44.942584 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:44.942754 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:44.943381 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:44.943397 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:44.945323 (+ 1926us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:44.947097 (+ 1774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:44.947106 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:44.947124 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:44.947517 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:44.947788 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:44.947794 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:44.947798 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:44.947985 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:44.948365 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:44.948369 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:44.948601 (+ 232us) client_negotiation.cc:769] Sending connection context
0808 18:22:44.948792 (+ 191us) client_negotiation.cc:240] Negotiation successful
0808 18:22:44.949044 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":178,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.130@KRBTEST.COM" created.
I20260808 18:22:44.951206 27342 heartbeater.cc:347] Connected to a master server at 127.23.72.190:38711
I20260808 18:22:44.951501 27342 heartbeater.cc:464] Registering TS with master...
I20260808 18:22:44.952083 27342 heartbeater.cc:511] Master 127.23.72.190:38711 requested a full tablet report, sending...
I20260808 18:22:44.953785 27135 ts_manager.cc:194] Registered new tserver with Master: 4667296435b348b68b5f1796aef6a38c (127.23.72.129:40971)
I20260808 18:22:44.954900 27135 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.129@KRBTEST.COM'} at 127.23.72.129:39181
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab.
Entry for principal kudu/127.23.72.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab.
I20260808 18:22:44.966650 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.130:0
--local_ip_for_outbound_sockets=127.23.72.130
--webserver_interface=127.23.72.130
--webserver_port=0
--tserver_master_addrs=127.23.72.190:38711
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab
--principal=kudu/127.23.72.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:42585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:45.079028 27347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:45.079339 27347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:45.079416 27347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:45.083616 27347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:45.083693 27347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:45.083781 27347 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.130
I20260808 18:22:45.089105 27347 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:42585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab
--principal=kudu/127.23.72.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.72.130
--webserver_port=0
--tserver_master_addrs=127.23.72.190:38711
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27347
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.130
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:45.090143 27347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:45.091120 27347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:45.100721 27352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:45.100752 27353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:45.100925 27347 server_base.cc:1034] running on GCE node
W20260808 18:22:45.101052 27355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:45.101455 27347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:45.102065 27347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:45.103252 27347 hybrid_clock.cc:648] HybridClock initialized: now 1786213365103224 us; error 39 us; skew 500 ppm
Aug 08 18:22:45 dist-test-slave-5k54 krb5kdc[27088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213365, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:45.106194 27347 init.cc:377] Logged in from keytab as kudu/127.23.72.130@KRBTEST.COM (short username kudu)
I20260808 18:22:45.107059 27347 webserver.cc:460] Webserver started at http://127.23.72.130:36121/ using document root <none> and password file <none>
I20260808 18:22:45.107648 27347 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:45.107741 27347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:45.107971 27347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:45.109985 27347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-1/data/instance:
uuid: "69d36f1a20ad4b9d8c6d3211a4ec5d68"
format_stamp: "Formatted at 2026-08-08 18:22:45 on dist-test-slave-5k54"
server_key: "a9044549580bd708e68a5aabdd0e3213"
server_key_iv: "5ccbbaeea0c03b286fe222ae683f8dff"
server_key_version: "encryptionkey@0"
I20260808 18:22:45.110566 27347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance:
uuid: "69d36f1a20ad4b9d8c6d3211a4ec5d68"
format_stamp: "Formatted at 2026-08-08 18:22:45 on dist-test-slave-5k54"
server_key: "a9044549580bd708e68a5aabdd0e3213"
server_key_iv: "5ccbbaeea0c03b286fe222ae683f8dff"
server_key_version: "encryptionkey@0"
I20260808 18:22:45.114609 27347 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260808 18:22:45.117168 27362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:45.118415 27347 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:22:45.118579 27347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-1/wal
uuid: "69d36f1a20ad4b9d8c6d3211a4ec5d68"
format_stamp: "Formatted at 2026-08-08 18:22:45 on dist-test-slave-5k54"
server_key: "a9044549580bd708e68a5aabdd0e3213"
server_key_iv: "5ccbbaeea0c03b286fe222ae683f8dff"
server_key_version: "encryptionkey@0"
I20260808 18:22:45.118696 27347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:45.133404 27347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:45.136690 27347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:45.136909 27347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:45.137482 27347 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:22:45.138275 27347 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:22:45.138347 27347 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:45.138419 27347 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:22:45.138473 27347 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:45.147629 27347 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.130:34205
I20260808 18:22:45.147666 27485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.130:34205 every 8 connection(s)
I20260808 18:22:45.148469 27347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
I20260808 18:22:45.153825 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 27347
I20260808 18:22:45.153975 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance
I20260808 18:22:45.154274 23842 external_mini_cluster.cc:1428] Setting key 832e6f637221fd22cca07081f7241839
Aug 08 18:22:45 dist-test-slave-5k54 krb5kdc[27088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213365, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.130@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:22:45.162819 27178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:45.150184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:52267 (local address 127.23.72.190:38711)
0808 18:22:45.150333 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:45.150339 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:45.151209 (+ 870us) server_negotiation.cc:407] Connection header received
0808 18:22:45.152564 (+ 1355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:45.152567 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:45.152614 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:45.152690 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:45.153796 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:45.154628 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:45.156166 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:45.156554 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:45.156679 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:45.159018 (+ 2339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:45.159021 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:45.159023 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:45.159044 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:45.160599 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:45.161225 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:45.161229 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:45.161230 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:45.161311 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:45.161655 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:45.161660 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:45.161661 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:45.161841 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:45.161965 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:45.162604 (+ 639us) server_negotiation.cc:299] Negotiation successful
0808 18:22:45.162701 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 18:22:45.164004 27488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:45.150605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:38711 (local address 127.23.72.130:52267)
0808 18:22:45.151054 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:45.151096 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:45.152357 (+ 1261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:45.152847 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:45.152853 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:45.153179 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:45.153635 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:45.153647 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:45.154767 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:45.154770 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:45.156044 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:45.156053 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:45.156697 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:45.156699 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:45.156836 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:45.157422 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:45.157437 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:45.158850 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:45.160713 (+ 1863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:45.160721 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:45.160738 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:45.161112 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:45.161417 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:45.161420 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:45.161422 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:45.161558 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:45.162011 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:45.162018 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:45.162335 (+ 317us) client_negotiation.cc:769] Sending connection context
0808 18:22:45.162563 (+ 228us) client_negotiation.cc:240] Negotiation successful
0808 18:22:45.162915 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":110,"threads_started":1}
WARNING: no policy specified for kudu/127.23.72.131@KRBTEST.COM; defaulting to no policy
I20260808 18:22:45.165195 27486 heartbeater.cc:347] Connected to a master server at 127.23.72.190:38711
I20260808 18:22:45.165442 27486 heartbeater.cc:464] Registering TS with master...
I20260808 18:22:45.165951 27486 heartbeater.cc:511] Master 127.23.72.190:38711 requested a full tablet report, sending...
I20260808 18:22:45.167089 27135 ts_manager.cc:194] Registered new tserver with Master: 69d36f1a20ad4b9d8c6d3211a4ec5d68 (127.23.72.130:34205)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.131@KRBTEST.COM" created.
I20260808 18:22:45.167834 27135 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.130@KRBTEST.COM'} at 127.23.72.130:52267
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab.
Entry for principal kudu/127.23.72.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab.
I20260808 18:22:45.183696 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.131:0
--local_ip_for_outbound_sockets=127.23.72.131
--webserver_interface=127.23.72.131
--webserver_port=0
--tserver_master_addrs=127.23.72.190:38711
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab
--principal=kudu/127.23.72.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:42585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:45.293998 27492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:45.294227 27492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:45.294269 27492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:45.298362 27492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:45.298420 27492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:45.298476 27492 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.131
I20260808 18:22:45.303606 27492 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:42585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab
--principal=kudu/127.23.72.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.72.131
--webserver_port=0
--tserver_master_addrs=127.23.72.190:38711
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.131
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:45.304486 27492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:45.305380 27492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:45.314919 27500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:45.314952 27497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:45.315042 27492 server_base.cc:1034] running on GCE node
W20260808 18:22:45.314986 27498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:45.315622 27492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:45.316255 27492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:45.317447 27492 hybrid_clock.cc:648] HybridClock initialized: now 1786213365317438 us; error 27 us; skew 500 ppm
Aug 08 18:22:45 dist-test-slave-5k54 krb5kdc[27088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213365, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:45.320175 27492 init.cc:377] Logged in from keytab as kudu/127.23.72.131@KRBTEST.COM (short username kudu)
I20260808 18:22:45.321102 27492 webserver.cc:460] Webserver started at http://127.23.72.131:46465/ using document root <none> and password file <none>
I20260808 18:22:45.321652 27492 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:45.321744 27492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:45.321996 27492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:45.324074 27492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-2/data/instance:
uuid: "d9b808d66ec746638e035e85805325ae"
format_stamp: "Formatted at 2026-08-08 18:22:45 on dist-test-slave-5k54"
server_key: "a16123fa2d0dac17d3e7777c80ebc21c"
server_key_iv: "51827aedf10a25aa7975c4b27e073dc2"
server_key_version: "encryptionkey@0"
I20260808 18:22:45.324637 27492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance:
uuid: "d9b808d66ec746638e035e85805325ae"
format_stamp: "Formatted at 2026-08-08 18:22:45 on dist-test-slave-5k54"
server_key: "a16123fa2d0dac17d3e7777c80ebc21c"
server_key_iv: "51827aedf10a25aa7975c4b27e073dc2"
server_key_version: "encryptionkey@0"
I20260808 18:22:45.328719 27492 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 18:22:45.331136 27507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:45.332410 27492 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:22:45.332545 27492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-2/wal
uuid: "d9b808d66ec746638e035e85805325ae"
format_stamp: "Formatted at 2026-08-08 18:22:45 on dist-test-slave-5k54"
server_key: "a16123fa2d0dac17d3e7777c80ebc21c"
server_key_iv: "51827aedf10a25aa7975c4b27e073dc2"
server_key_version: "encryptionkey@0"
I20260808 18:22:45.332659 27492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:45.349335 27492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:45.352066 27492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:45.352293 27492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:45.352859 27492 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:22:45.353660 27492 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:22:45.353730 27492 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:45.353802 27492 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:22:45.353844 27492 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:45.363268 27492 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.131:42835
I20260808 18:22:45.363292 27634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.131:42835 every 8 connection(s)
I20260808 18:22:45.364182 27492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
I20260808 18:22:45.370409 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 27492
I20260808 18:22:45.370539 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance
I20260808 18:22:45.370841 23842 external_mini_cluster.cc:1428] Setting key 8b4b09d00727863df9cd5d56aac1e836
Aug 08 18:22:45 dist-test-slave-5k54 krb5kdc[27088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213365, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.131@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:22:45.378060 27178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:45.365931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:41873 (local address 127.23.72.190:38711)
0808 18:22:45.366069 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:45.366073 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:45.366913 (+ 840us) server_negotiation.cc:407] Connection header received
0808 18:22:45.368301 (+ 1388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:45.368304 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:45.368365 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:45.368472 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:45.369640 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:45.370223 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:45.371734 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:45.371966 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:45.372074 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:45.374559 (+ 2485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:45.374563 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:45.374565 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:45.374593 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:45.376051 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:45.376560 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:45.376564 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:45.376566 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:45.376646 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:45.376987 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:45.376991 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:45.376993 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:45.377174 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:45.377282 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:45.377759 (+ 477us) server_negotiation.cc:299] Negotiation successful
0808 18:22:45.377878 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 18:22:45.378808 27637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:45.366320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:38711 (local address 127.23.72.131:41873)
0808 18:22:45.366765 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:45.366805 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:45.368095 (+ 1290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:45.368639 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:45.368645 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:45.368988 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:45.369491 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:45.369506 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:45.370347 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:45.370350 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:45.371611 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:45.371621 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:45.372110 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:45.372114 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:45.372294 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:45.372968 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:45.372983 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:45.374390 (+ 1407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:45.376179 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:45.376184 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:45.376198 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:45.376457 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:45.376744 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:45.376748 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:45.376750 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:45.376890 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:45.377279 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:45.377285 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:45.377531 (+ 246us) client_negotiation.cc:769] Sending connection context
0808 18:22:45.377716 (+ 185us) client_negotiation.cc:240] Negotiation successful
0808 18:22:45.378020 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":109,"threads_started":1}
I20260808 18:22:45.379966 27635 heartbeater.cc:347] Connected to a master server at 127.23.72.190:38711
I20260808 18:22:45.380174 27635 heartbeater.cc:464] Registering TS with master...
I20260808 18:22:45.380580 27635 heartbeater.cc:511] Master 127.23.72.190:38711 requested a full tablet report, sending...
I20260808 18:22:45.381508 27135 ts_manager.cc:194] Registered new tserver with Master: d9b808d66ec746638e035e85805325ae (127.23.72.131:42835)
I20260808 18:22:45.382017 27135 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.131@KRBTEST.COM'} at 127.23.72.131:41873
I20260808 18:22:45.384781 23842 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 18:22:45 dist-test-slave-5k54 krb5kdc[27088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213365, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 08 18:22:45 dist-test-slave-5k54 krb5kdc[27088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213365, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:22:45.411582 27178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:45.403301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42468 (local address 127.23.72.190:38711)
0808 18:22:45.403516 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:45.403520 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:45.403606 (+ 86us) server_negotiation.cc:407] Connection header received
0808 18:22:45.403719 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:45.403721 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:45.403766 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:45.403835 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:45.404504 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:45.405034 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:45.405807 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:45.406018 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:45.406092 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:45.408578 (+ 2486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:45.408581 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:45.408583 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:45.408605 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:45.409838 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:45.410292 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:45.410297 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:45.410298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:45.410387 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:45.410663 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:45.410667 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:45.410668 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:45.410899 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:45.411042 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:45.411272 (+ 230us) server_negotiation.cc:299] Negotiation successful
0808 18:22:45.411388 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
W20260808 18:22:45.412017 27135 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:42468
I20260808 18:22:45.412889 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 27201
I20260808 18:22:45.420459 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 27347
I20260808 18:22:45.428879 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 27492
I20260808 18:22:45.436172 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 27102
2026-08-08T18:22:45Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2196 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:22:45 dist-test-slave-5k54 krb5kdc[27647](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:22:45 dist-test-slave-5k54 krb5kdc[27647](info): set up 2 sockets
Aug 08 18:22:45 dist-test-slave-5k54 krb5kdc[27647](info): commencing operation
krb5kdc: starting...
W20260808 18:22:46.512110 23842 mini_kdc.cc:121] Time spent starting KDC: real 1.053s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Aug 08 18:22:46 dist-test-slave-5k54 krb5kdc[27647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:22:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:22:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.72.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
I20260808 18:22:46.636565 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:46077
--webserver_interface=127.23.72.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:39681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:46077
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:46.747902 27660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:46.748160 27660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:46.748200 27660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:46.752184 27660 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:22:46.752234 27660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:46.752254 27660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:46.752270 27660 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:22:46.752285 27660 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:22:46.757328 27660 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:39681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:46077
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:46077
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.72.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27660
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:46.758313 27660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:46.759297 27660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:46.767464 27667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:46.767501 27670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:46.767530 27668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:46.767652 27660 server_base.cc:1034] running on GCE node
I20260808 18:22:46.768240 27660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:46.769280 27660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:46.770494 27660 hybrid_clock.cc:648] HybridClock initialized: now 1786213366770464 us; error 39 us; skew 500 ppm
Aug 08 18:22:46 dist-test-slave-5k54 krb5kdc[27647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213366, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:46.773675 27660 init.cc:377] Logged in from keytab as kudu/127.23.72.190@KRBTEST.COM (short username kudu)
I20260808 18:22:46.774557 27660 webserver.cc:460] Webserver started at http://127.23.72.190:44591/ using document root <none> and password file <none>
I20260808 18:22:46.775098 27660 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:46.775161 27660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:46.775337 27660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:46.777487 27660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "5ff21865aac64fbba16b88585bb16e4f"
format_stamp: "Formatted at 2026-08-08 18:22:46 on dist-test-slave-5k54"
server_key: "757cecd4421f84b2b124d8fb39e86e2c"
server_key_iv: "773dec55483ba28b2ac980c91beddce7"
server_key_version: "encryptionkey@0"
I20260808 18:22:46.778040 27660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "5ff21865aac64fbba16b88585bb16e4f"
format_stamp: "Formatted at 2026-08-08 18:22:46 on dist-test-slave-5k54"
server_key: "757cecd4421f84b2b124d8fb39e86e2c"
server_key_iv: "773dec55483ba28b2ac980c91beddce7"
server_key_version: "encryptionkey@0"
I20260808 18:22:46.781906 27660 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 18:22:46.784468 27677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:46.785745 27660 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:22:46.785876 27660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "5ff21865aac64fbba16b88585bb16e4f"
format_stamp: "Formatted at 2026-08-08 18:22:46 on dist-test-slave-5k54"
server_key: "757cecd4421f84b2b124d8fb39e86e2c"
server_key_iv: "773dec55483ba28b2ac980c91beddce7"
server_key_version: "encryptionkey@0"
I20260808 18:22:46.785997 27660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:46.809296 27660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:46.812644 27660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:46.812847 27660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:46.820041 27660 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.190:46077
I20260808 18:22:46.820065 27733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.190:46077 every 8 connection(s)
I20260808 18:22:46.820966 27660 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:22:46.822801 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 27660
I20260808 18:22:46.822947 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:22:46.823246 23842 external_mini_cluster.cc:1428] Setting key 5f56c6fe6835ae989b0ef2d113c24406
I20260808 18:22:46.823897 27734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:46.831173 27734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ff21865aac64fbba16b88585bb16e4f: Bootstrap starting.
Aug 08 18:22:46 dist-test-slave-5k54 krb5kdc[27647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:22:46.833726 27734 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ff21865aac64fbba16b88585bb16e4f: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:46.834463 27734 log.cc:824] T 00000000000000000000000000000000 P 5ff21865aac64fbba16b88585bb16e4f: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:46.836815 27734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ff21865aac64fbba16b88585bb16e4f: No bootstrap required, opened a new log
I20260808 18:22:46.838845 27738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:46.824744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58592 (local address 127.23.72.190:46077)
0808 18:22:46.825197 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:46.825216 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:46.825260 (+ 44us) server_negotiation.cc:407] Connection header received
0808 18:22:46.826076 (+ 816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:46.826099 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:46.826510 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:46.826886 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:46.828008 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:46.829491 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:46.830346 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:46.831032 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:46.831202 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:46.833948 (+ 2746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:46.833954 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:46.833977 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:46.834006 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:46.836074 (+ 2068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:46.836502 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:46.836507 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:46.836513 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:46.836654 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:46.836965 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:46.836969 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:46.836971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:46.837356 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:46.837574 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:46.837872 (+ 298us) server_negotiation.cc:299] Negotiation successful
0808 18:22:46.838153 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":138,"threads_started":1}
I20260808 18:22:46.840687 27734 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ff21865aac64fbba16b88585bb16e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ff21865aac64fbba16b88585bb16e4f" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 46077 } }
I20260808 18:22:46.840953 27734 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5ff21865aac64fbba16b88585bb16e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:46.841015 27734 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ff21865aac64fbba16b88585bb16e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ff21865aac64fbba16b88585bb16e4f, State: Initialized, Role: FOLLOWER
I20260808 18:22:46.841482 27734 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ff21865aac64fbba16b88585bb16e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ff21865aac64fbba16b88585bb16e4f" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 46077 } }
I20260808 18:22:46.841641 27734 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5ff21865aac64fbba16b88585bb16e4f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:22:46.841691 27734 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ff21865aac64fbba16b88585bb16e4f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:22:46.841761 27734 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ff21865aac64fbba16b88585bb16e4f [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:46.842619 27734 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ff21865aac64fbba16b88585bb16e4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ff21865aac64fbba16b88585bb16e4f" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 46077 } }
I20260808 18:22:46.842905 27734 leader_election.cc:302] T 00000000000000000000000000000000 P 5ff21865aac64fbba16b88585bb16e4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5ff21865aac64fbba16b88585bb16e4f; no voters:
I20260808 18:22:46.843183 27734 leader_election.cc:288] T 00000000000000000000000000000000 P 5ff21865aac64fbba16b88585bb16e4f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:22:46.843325 27740 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ff21865aac64fbba16b88585bb16e4f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:22:46.843616 27740 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5ff21865aac64fbba16b88585bb16e4f [term 1 LEADER]: Becoming Leader. State: Replica: 5ff21865aac64fbba16b88585bb16e4f, State: Running, Role: LEADER
I20260808 18:22:46.843974 27740 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5ff21865aac64fbba16b88585bb16e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ff21865aac64fbba16b88585bb16e4f" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 46077 } }
I20260808 18:22:46.844432 27734 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ff21865aac64fbba16b88585bb16e4f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:22:46.845737 27741 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ff21865aac64fbba16b88585bb16e4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5ff21865aac64fbba16b88585bb16e4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ff21865aac64fbba16b88585bb16e4f" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 46077 } } }
I20260808 18:22:46.845893 27741 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ff21865aac64fbba16b88585bb16e4f [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:46.846122 27742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ff21865aac64fbba16b88585bb16e4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ff21865aac64fbba16b88585bb16e4f. Latest consensus state: current_term: 1 leader_uuid: "5ff21865aac64fbba16b88585bb16e4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ff21865aac64fbba16b88585bb16e4f" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 46077 } } }
I20260808 18:22:46.846220 27742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ff21865aac64fbba16b88585bb16e4f [sys.catalog]: This master's current role is: LEADER
W20260808 18:22:46.849148 27756 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5ff21865aac64fbba16b88585bb16e4f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:22:46.849236 27756 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:22:46.849318 27749 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:22:46.850026 27749 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:22:46.855670 27749 catalog_manager.cc:1347] Generated new cluster ID: 33e80b8f282e42b7bf062007fd76bfdb
I20260808 18:22:46.855738 27749 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:22:46.873513 27749 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:22:46.874730 27749 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:22:46.884264 27749 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5ff21865aac64fbba16b88585bb16e4f: Generated new TSK 0
I20260808 18:22:46.885028 27749 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.72.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab.
Entry for principal kudu/127.23.72.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab.
I20260808 18:22:46.922257 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.129:0
--local_ip_for_outbound_sockets=127.23.72.129
--webserver_interface=127.23.72.129
--webserver_port=0
--tserver_master_addrs=127.23.72.190:46077
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab
--principal=kudu/127.23.72.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:39681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:47.058921 27762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:47.059162 27762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:47.059203 27762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:47.063126 27762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:47.063176 27762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:47.063230 27762 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.129
I20260808 18:22:47.068480 27762 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:39681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab
--principal=kudu/127.23.72.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.72.129
--webserver_port=0
--tserver_master_addrs=127.23.72.190:46077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.129
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:47.069480 27762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:47.070425 27762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:47.079906 27769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:47.079943 27768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:47.079908 27771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:47.080353 27762 server_base.cc:1034] running on GCE node
I20260808 18:22:47.080780 27762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:47.081404 27762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:47.082592 27762 hybrid_clock.cc:648] HybridClock initialized: now 1786213367082560 us; error 44 us; skew 500 ppm
Aug 08 18:22:47 dist-test-slave-5k54 krb5kdc[27647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213367, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:47.085464 27762 init.cc:377] Logged in from keytab as kudu/127.23.72.129@KRBTEST.COM (short username kudu)
I20260808 18:22:47.086366 27762 webserver.cc:460] Webserver started at http://127.23.72.129:39975/ using document root <none> and password file <none>
I20260808 18:22:47.086871 27762 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:47.086931 27762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:47.087095 27762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:47.089244 27762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-0/data/instance:
uuid: "3ba445397d9e4e948fb0c5b905ec429b"
format_stamp: "Formatted at 2026-08-08 18:22:47 on dist-test-slave-5k54"
server_key: "aeff96611278311efa10ac59148f3859"
server_key_iv: "0fad9c263997cc10623e2fc335e06b8b"
server_key_version: "encryptionkey@0"
I20260808 18:22:47.089818 27762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance:
uuid: "3ba445397d9e4e948fb0c5b905ec429b"
format_stamp: "Formatted at 2026-08-08 18:22:47 on dist-test-slave-5k54"
server_key: "aeff96611278311efa10ac59148f3859"
server_key_iv: "0fad9c263997cc10623e2fc335e06b8b"
server_key_version: "encryptionkey@0"
I20260808 18:22:47.093509 27762 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 18:22:47.095944 27780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:47.097093 27762 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:22:47.097227 27762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-0/wal
uuid: "3ba445397d9e4e948fb0c5b905ec429b"
format_stamp: "Formatted at 2026-08-08 18:22:47 on dist-test-slave-5k54"
server_key: "aeff96611278311efa10ac59148f3859"
server_key_iv: "0fad9c263997cc10623e2fc335e06b8b"
server_key_version: "encryptionkey@0"
I20260808 18:22:47.097344 27762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:47.115406 27762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:47.118335 27762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:47.118541 27762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:47.119132 27762 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:22:47.120035 27762 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:22:47.120106 27762 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:47.120179 27762 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:22:47.120232 27762 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:47.129280 27762 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.129:40117
I20260808 18:22:47.129299 27905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.129:40117 every 8 connection(s)
I20260808 18:22:47.130235 27762 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
Aug 08 18:22:47 dist-test-slave-5k54 krb5kdc[27647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213367, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:22:47.139983 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 27762
I20260808 18:22:47.140110 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance
I20260808 18:22:47.140419 23842 external_mini_cluster.cc:1428] Setting key 84d5bc4b38521b34d03a86733ea51273
I20260808 18:22:47.144161 27738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:47.132065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:52015 (local address 127.23.72.190:46077)
0808 18:22:47.132208 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:47.132212 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:47.133057 (+ 845us) server_negotiation.cc:407] Connection header received
0808 18:22:47.134468 (+ 1411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:47.134471 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:47.134533 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:47.134623 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:47.135832 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:47.136380 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:47.137534 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:47.137814 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:47.137935 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:47.140469 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:47.140472 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:47.140474 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:47.140519 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:47.142098 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:47.142596 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:47.142601 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:47.142603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:47.142685 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:47.143036 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:47.143039 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:47.143041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:47.143219 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:47.143337 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:47.143869 (+ 532us) server_negotiation.cc:299] Negotiation successful
0808 18:22:47.143993 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 18:22:47.144996 27908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:47.132445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46077 (local address 127.23.72.129:52015)
0808 18:22:47.132903 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:47.132946 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:47.134246 (+ 1300us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:47.134812 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:47.134819 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:47.135165 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:47.135669 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:47.135682 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:47.136515 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:47.136523 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:47.137399 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:47.137409 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:47.137986 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:47.137990 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:47.138148 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:47.138673 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:47.138689 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:47.140300 (+ 1611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:47.142213 (+ 1913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:47.142218 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:47.142232 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:47.142491 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:47.142779 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:47.142782 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:47.142784 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:47.142934 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:47.143330 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:47.143336 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:47.143633 (+ 297us) client_negotiation.cc:769] Sending connection context
0808 18:22:47.143833 (+ 200us) client_negotiation.cc:240] Negotiation successful
0808 18:22:47.144142 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":104,"threads_started":1}
I20260808 18:22:47.146164 27906 heartbeater.cc:347] Connected to a master server at 127.23.72.190:46077
I20260808 18:22:47.146406 27906 heartbeater.cc:464] Registering TS with master...
I20260808 18:22:47.147039 27906 heartbeater.cc:511] Master 127.23.72.190:46077 requested a full tablet report, sending...
I20260808 18:22:47.148571 27695 ts_manager.cc:194] Registered new tserver with Master: 3ba445397d9e4e948fb0c5b905ec429b (127.23.72.129:40117)
I20260808 18:22:47.149667 27695 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.129@KRBTEST.COM'} at 127.23.72.129:52015
WARNING: no policy specified for kudu/127.23.72.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab.
Entry for principal kudu/127.23.72.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab.
I20260808 18:22:47.169855 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.130:0
--local_ip_for_outbound_sockets=127.23.72.130
--webserver_interface=127.23.72.130
--webserver_port=0
--tserver_master_addrs=127.23.72.190:46077
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab
--principal=kudu/127.23.72.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:39681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:47.277938 27911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:47.278201 27911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:47.278244 27911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:47.282188 27911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:47.282250 27911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:47.282302 27911 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.130
I20260808 18:22:47.287531 27911 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:39681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab
--principal=kudu/127.23.72.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.72.130
--webserver_port=0
--tserver_master_addrs=127.23.72.190:46077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.130
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:47.288475 27911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:47.289376 27911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:47.298724 27916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:47.298760 27911 server_base.cc:1034] running on GCE node
W20260808 18:22:47.298728 27917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:47.298718 27919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:47.299279 27911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:47.299906 27911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:47.301128 27911 hybrid_clock.cc:648] HybridClock initialized: now 1786213367301079 us; error 44 us; skew 500 ppm
Aug 08 18:22:47 dist-test-slave-5k54 krb5kdc[27647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213367, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:47.304014 27911 init.cc:377] Logged in from keytab as kudu/127.23.72.130@KRBTEST.COM (short username kudu)
I20260808 18:22:47.304864 27911 webserver.cc:460] Webserver started at http://127.23.72.130:35733/ using document root <none> and password file <none>
I20260808 18:22:47.305533 27911 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:47.305601 27911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:47.305856 27911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:47.307911 27911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-1/data/instance:
uuid: "40dbdce0b8ec49a08f8708f291dd6e01"
format_stamp: "Formatted at 2026-08-08 18:22:47 on dist-test-slave-5k54"
server_key: "f38d0d024103d5d9b19fe9e353e63800"
server_key_iv: "17f572b389027ab8f79bd3d0d91d254a"
server_key_version: "encryptionkey@0"
I20260808 18:22:47.308449 27911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance:
uuid: "40dbdce0b8ec49a08f8708f291dd6e01"
format_stamp: "Formatted at 2026-08-08 18:22:47 on dist-test-slave-5k54"
server_key: "f38d0d024103d5d9b19fe9e353e63800"
server_key_iv: "17f572b389027ab8f79bd3d0d91d254a"
server_key_version: "encryptionkey@0"
I20260808 18:22:47.312230 27911 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 18:22:47.314522 27926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:47.316035 27911 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:22:47.316161 27911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-1/wal
uuid: "40dbdce0b8ec49a08f8708f291dd6e01"
format_stamp: "Formatted at 2026-08-08 18:22:47 on dist-test-slave-5k54"
server_key: "f38d0d024103d5d9b19fe9e353e63800"
server_key_iv: "17f572b389027ab8f79bd3d0d91d254a"
server_key_version: "encryptionkey@0"
I20260808 18:22:47.316279 27911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:47.338517 27911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:47.342087 27911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:47.342283 27911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:47.342851 27911 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:22:47.343755 27911 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:22:47.343820 27911 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:47.343904 27911 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:22:47.343955 27911 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:47.353664 27911 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.130:45529
I20260808 18:22:47.353686 28049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.130:45529 every 8 connection(s)
I20260808 18:22:47.354622 27911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
I20260808 18:22:47.356248 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 27911
I20260808 18:22:47.356386 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance
I20260808 18:22:47.356753 23842 external_mini_cluster.cc:1428] Setting key d9a727286b29fff39bb5c3c979cc122a
Aug 08 18:22:47 dist-test-slave-5k54 krb5kdc[27647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213367, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.130@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.72.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.131@KRBTEST.COM" created.
I20260808 18:22:47.370522 27738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:47.356540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:60185 (local address 127.23.72.190:46077)
0808 18:22:47.356698 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:47.356702 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:47.357700 (+ 998us) server_negotiation.cc:407] Connection header received
0808 18:22:47.359507 (+ 1807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:47.359510 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:47.359561 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:47.359637 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:47.361226 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:47.362183 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:47.364078 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:47.364369 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:47.364495 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:47.366954 (+ 2459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:47.366958 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:47.366960 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:47.366995 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:47.368506 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:47.369017 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:47.369020 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:47.369022 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:47.369107 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:47.369453 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:47.369458 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:47.369460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:47.369638 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:47.369744 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:47.370222 (+ 478us) server_negotiation.cc:299] Negotiation successful
0808 18:22:47.370346 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 18:22:47.371301 28052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:47.357034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46077 (local address 127.23.72.130:60185)
0808 18:22:47.357538 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:47.357590 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:47.359232 (+ 1642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:47.359834 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:47.359845 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:47.360361 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:47.361058 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:47.361073 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:47.362352 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:47.362358 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:47.363913 (+ 1555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:47.363928 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:47.364533 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:47.364537 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:47.364713 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:47.365333 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:47.365349 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:47.366770 (+ 1421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:47.368632 (+ 1862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:47.368637 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:47.368653 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:47.368911 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:47.369202 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:47.369206 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:47.369208 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:47.369348 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:47.369742 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:47.369749 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:47.369993 (+ 244us) client_negotiation.cc:769] Sending connection context
0808 18:22:47.370178 (+ 185us) client_negotiation.cc:240] Negotiation successful
0808 18:22:47.370499 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":144,"threads_started":1}
I20260808 18:22:47.372643 28050 heartbeater.cc:347] Connected to a master server at 127.23.72.190:46077
I20260808 18:22:47.372905 28050 heartbeater.cc:464] Registering TS with master...
I20260808 18:22:47.373499 28050 heartbeater.cc:511] Master 127.23.72.190:46077 requested a full tablet report, sending...
I20260808 18:22:47.374636 27695 ts_manager.cc:194] Registered new tserver with Master: 40dbdce0b8ec49a08f8708f291dd6e01 (127.23.72.130:45529)
I20260808 18:22:47.375205 27695 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.130@KRBTEST.COM'} at 127.23.72.130:60185
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab.
Entry for principal kudu/127.23.72.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab.
I20260808 18:22:47.386729 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.131:0
--local_ip_for_outbound_sockets=127.23.72.131
--webserver_interface=127.23.72.131
--webserver_port=0
--tserver_master_addrs=127.23.72.190:46077
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab
--principal=kudu/127.23.72.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:39681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:47.502532 28055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:47.502799 28055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:47.502846 28055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:47.507283 28055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:47.507340 28055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:47.507405 28055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.131
I20260808 18:22:47.513370 28055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:39681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab
--principal=kudu/127.23.72.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.72.131
--webserver_port=0
--tserver_master_addrs=127.23.72.190:46077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28055
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.131
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:47.514416 28055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:47.515594 28055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:47.525632 28061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:47.525684 28060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:47.525684 28063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:47.525897 28055 server_base.cc:1034] running on GCE node
I20260808 18:22:47.526362 28055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:47.526990 28055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:47.528147 28055 hybrid_clock.cc:648] HybridClock initialized: now 1786213367528134 us; error 24 us; skew 500 ppm
Aug 08 18:22:47 dist-test-slave-5k54 krb5kdc[27647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213367, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:47.530936 28055 init.cc:377] Logged in from keytab as kudu/127.23.72.131@KRBTEST.COM (short username kudu)
I20260808 18:22:47.531821 28055 webserver.cc:460] Webserver started at http://127.23.72.131:36509/ using document root <none> and password file <none>
I20260808 18:22:47.532375 28055 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:47.532478 28055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:47.532712 28055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:47.534709 28055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-2/data/instance:
uuid: "347860819d28426f89d6d158c2723c9b"
format_stamp: "Formatted at 2026-08-08 18:22:47 on dist-test-slave-5k54"
server_key: "5858f22713a7ba684895267dfce908b9"
server_key_iv: "751db7d445c0c46f8de6925e0e0d783f"
server_key_version: "encryptionkey@0"
I20260808 18:22:47.535271 28055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance:
uuid: "347860819d28426f89d6d158c2723c9b"
format_stamp: "Formatted at 2026-08-08 18:22:47 on dist-test-slave-5k54"
server_key: "5858f22713a7ba684895267dfce908b9"
server_key_iv: "751db7d445c0c46f8de6925e0e0d783f"
server_key_version: "encryptionkey@0"
I20260808 18:22:47.538883 28055 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 18:22:47.541319 28071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:47.542512 28055 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:22:47.542647 28055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-2/wal
uuid: "347860819d28426f89d6d158c2723c9b"
format_stamp: "Formatted at 2026-08-08 18:22:47 on dist-test-slave-5k54"
server_key: "5858f22713a7ba684895267dfce908b9"
server_key_iv: "751db7d445c0c46f8de6925e0e0d783f"
server_key_version: "encryptionkey@0"
I20260808 18:22:47.542758 28055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:47.557018 28055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:47.560317 28055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:47.560528 28055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:47.561100 28055 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:22:47.561915 28055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:22:47.561983 28055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:47.562052 28055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:22:47.562103 28055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:47.570946 28055 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.131:44689
I20260808 18:22:47.570982 28195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.131:44689 every 8 connection(s)
I20260808 18:22:47.571889 28055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
I20260808 18:22:47.573957 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 28055
I20260808 18:22:47.574120 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance
I20260808 18:22:47.574424 23842 external_mini_cluster.cc:1428] Setting key 7272d80d398d904262bf0c57d6c32293
Aug 08 18:22:47 dist-test-slave-5k54 krb5kdc[27647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213367, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.131@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:22:47.585261 27738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:47.573641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:60457 (local address 127.23.72.190:46077)
0808 18:22:47.573817 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:47.573822 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:47.574756 (+ 934us) server_negotiation.cc:407] Connection header received
0808 18:22:47.576349 (+ 1593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:47.576351 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:47.576397 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:47.576471 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:47.577642 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:47.578217 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:47.579335 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:47.579613 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:47.579697 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:47.582150 (+ 2453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:47.582153 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:47.582155 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:47.582179 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:47.583472 (+ 1293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:47.583939 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:47.583941 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:47.583943 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:47.584031 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:47.584334 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:47.584336 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:47.584337 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:47.584508 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:47.584586 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:47.585043 (+ 457us) server_negotiation.cc:299] Negotiation successful
0808 18:22:47.585142 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 18:22:47.586056 28198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:47.574047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46077 (local address 127.23.72.131:60457)
0808 18:22:47.574567 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:47.574627 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:47.576127 (+ 1500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:47.576670 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:47.576680 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:47.577054 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:47.577508 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:47.577521 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:47.578325 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:47.578327 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:47.579217 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:47.579226 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:47.579744 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:47.579748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:47.579884 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:47.580507 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:47.580523 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:47.582003 (+ 1480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:47.583575 (+ 1572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:47.583579 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:47.583590 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:47.583829 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:47.584121 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:47.584123 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:47.584124 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:47.584253 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:47.584595 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:47.584599 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:47.584821 (+ 222us) client_negotiation.cc:769] Sending connection context
0808 18:22:47.584987 (+ 166us) client_negotiation.cc:240] Negotiation successful
0808 18:22:47.585218 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":163,"threads_started":1}
I20260808 18:22:47.587229 28196 heartbeater.cc:347] Connected to a master server at 127.23.72.190:46077
I20260808 18:22:47.587525 28196 heartbeater.cc:464] Registering TS with master...
I20260808 18:22:47.588090 28196 heartbeater.cc:511] Master 127.23.72.190:46077 requested a full tablet report, sending...
I20260808 18:22:47.589121 27695 ts_manager.cc:194] Registered new tserver with Master: 347860819d28426f89d6d158c2723c9b (127.23.72.131:44689)
I20260808 18:22:47.589671 27695 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.131@KRBTEST.COM'} at 127.23.72.131:60457
I20260808 18:22:47.600446 23842 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 18:22:47 dist-test-slave-5k54 krb5kdc[27647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 18:22:47 dist-test-slave-5k54 krb5kdc[27647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.72.129@KRBTEST.COM
I20260808 18:22:47.628755 28203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:47.618374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59068 (local address 127.23.72.129:40117)
0808 18:22:47.618701 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:47.618717 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:47.618743 (+ 26us) server_negotiation.cc:407] Connection header received
0808 18:22:47.618845 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:47.618850 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:47.619057 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:47.619179 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:47.619963 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:47.620962 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:47.621789 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:47.622240 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:47.622339 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:47.624946 (+ 2607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:47.624955 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:47.624969 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:47.624996 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:47.626563 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:47.627164 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:47.627170 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:47.627172 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:47.627260 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:47.627657 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:47.627661 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:47.627663 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:47.627924 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:47.628156 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:47.628403 (+ 247us) server_negotiation.cc:299] Negotiation successful
0808 18:22:47.628571 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":94,"threads_started":1}
Aug 08 18:22:47 dist-test-slave-5k54 krb5kdc[27647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:22:47.640061 27738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:47.631980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58606 (local address 127.23.72.190:46077)
0808 18:22:47.632135 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:47.632141 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:47.632198 (+ 57us) server_negotiation.cc:407] Connection header received
0808 18:22:47.632346 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:47.632349 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:47.632433 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:47.632528 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:47.633350 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:47.633848 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:47.634594 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:47.634794 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:47.634866 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:47.637116 (+ 2250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:47.637119 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:47.637121 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:47.637142 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:47.638567 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:47.638924 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:47.638927 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:47.638928 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:47.638996 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:47.639215 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:47.639217 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:47.639218 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:47.639417 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:47.639575 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:47.639750 (+ 175us) server_negotiation.cc:299] Negotiation successful
0808 18:22:47.639844 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260808 18:22:47.640468 27695 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58606
I20260808 18:22:47.641126 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 27762
I20260808 18:22:47.649639 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 27911
I20260808 18:22:47.657539 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 28055
I20260808 18:22:47.663666 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 27660
2026-08-08T18:22:47Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2228 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:22:47 dist-test-slave-5k54 krb5kdc[28208](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:22:47 dist-test-slave-5k54 krb5kdc[28208](info): set up 2 sockets
Aug 08 18:22:47 dist-test-slave-5k54 krb5kdc[28208](info): commencing operation
krb5kdc: starting...
W20260808 18:22:48.731900 23842 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Aug 08 18:22:48 dist-test-slave-5k54 krb5kdc[28208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:22:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:22:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.72.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
I20260808 18:22:48.858215 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:37869
--webserver_interface=127.23.72.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:35927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:37869
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:48.972777 28222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:48.973019 28222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:48.973090 28222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:48.977095 28222 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:22:48.977149 28222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:48.977169 28222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:48.977211 28222 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:22:48.977244 28222 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:22:48.982846 28222 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:35927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:37869
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:37869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.72.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:48.983924 28222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:48.984928 28222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:48.992951 28228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:48.992997 28231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:48.992961 28229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:48.993177 28222 server_base.cc:1034] running on GCE node
I20260808 18:22:48.993803 28222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:48.994830 28222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:48.996032 28222 hybrid_clock.cc:648] HybridClock initialized: now 1786213368996019 us; error 35 us; skew 500 ppm
Aug 08 18:22:48 dist-test-slave-5k54 krb5kdc[28208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213368, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:48.998773 28222 init.cc:377] Logged in from keytab as kudu/127.23.72.190@KRBTEST.COM (short username kudu)
I20260808 18:22:48.999501 28222 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:48.999570 28222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:48.999737 28222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:49.001674 28222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "792615d15172496882a2f1dc04984ede"
format_stamp: "Formatted at 2026-08-08 18:22:48 on dist-test-slave-5k54"
server_key: "30a8448886f95337e0e515d606946814"
server_key_iv: "78aa23332861321c5b07b7b49e52c4ee"
server_key_version: "encryptionkey@0"
I20260808 18:22:49.002189 28222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "792615d15172496882a2f1dc04984ede"
format_stamp: "Formatted at 2026-08-08 18:22:48 on dist-test-slave-5k54"
server_key: "30a8448886f95337e0e515d606946814"
server_key_iv: "78aa23332861321c5b07b7b49e52c4ee"
server_key_version: "encryptionkey@0"
I20260808 18:22:49.005697 28222 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 18:22:49.008006 28237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:49.009262 28222 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:22:49.009418 28222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "792615d15172496882a2f1dc04984ede"
format_stamp: "Formatted at 2026-08-08 18:22:48 on dist-test-slave-5k54"
server_key: "30a8448886f95337e0e515d606946814"
server_key_iv: "78aa23332861321c5b07b7b49e52c4ee"
server_key_version: "encryptionkey@0"
I20260808 18:22:49.009524 28222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:49.026641 28222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:49.030531 28222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:49.030742 28222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:49.040669 28222 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.190:37869
I20260808 18:22:49.041584 28222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:22:49.043840 28295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.190:37869 every 8 connection(s)
I20260808 18:22:49.044476 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 28222
I20260808 18:22:49.044620 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:22:49.044961 23842 external_mini_cluster.cc:1428] Setting key 1a826ea2acd3791dcacf3ffc2cbe423e
I20260808 18:22:49.045902 28296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:49.053186 28296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 792615d15172496882a2f1dc04984ede: Bootstrap starting.
Aug 08 18:22:49 dist-test-slave-5k54 krb5kdc[28208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:22:49.055716 28296 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 792615d15172496882a2f1dc04984ede: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:49.056558 28296 log.cc:824] T 00000000000000000000000000000000 P 792615d15172496882a2f1dc04984ede: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:49.058436 28296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 792615d15172496882a2f1dc04984ede: No bootstrap required, opened a new log
I20260808 18:22:49.059738 28299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:49.046688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40822 (local address 127.23.72.190:37869)
0808 18:22:49.047070 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:49.047085 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:49.047120 (+ 35us) server_negotiation.cc:407] Connection header received
0808 18:22:49.047850 (+ 730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:49.047868 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:49.048180 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:49.048458 (+ 278us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:49.049384 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:49.050360 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:49.051635 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:49.052118 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:49.052239 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:49.055026 (+ 2787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:49.055030 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:49.055042 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:49.055070 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:49.056940 (+ 1870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:49.057421 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:49.057426 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:49.057430 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:49.057540 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:49.057828 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:49.057832 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:49.057834 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:49.058219 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:49.058423 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:49.058688 (+ 265us) server_negotiation.cc:299] Negotiation successful
0808 18:22:49.058972 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":118,"threads_started":1}
I20260808 18:22:49.062477 28296 raft_consensus.cc:348] T 00000000000000000000000000000000 P 792615d15172496882a2f1dc04984ede [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "792615d15172496882a2f1dc04984ede" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 37869 } }
I20260808 18:22:49.062706 28296 raft_consensus.cc:374] T 00000000000000000000000000000000 P 792615d15172496882a2f1dc04984ede [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:49.062806 28296 raft_consensus.cc:729] T 00000000000000000000000000000000 P 792615d15172496882a2f1dc04984ede [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792615d15172496882a2f1dc04984ede, State: Initialized, Role: FOLLOWER
I20260808 18:22:49.063247 28296 consensus_queue.cc:260] T 00000000000000000000000000000000 P 792615d15172496882a2f1dc04984ede [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "792615d15172496882a2f1dc04984ede" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 37869 } }
I20260808 18:22:49.063441 28296 raft_consensus.cc:388] T 00000000000000000000000000000000 P 792615d15172496882a2f1dc04984ede [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:22:49.063534 28296 raft_consensus.cc:482] T 00000000000000000000000000000000 P 792615d15172496882a2f1dc04984ede [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:22:49.063634 28296 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 792615d15172496882a2f1dc04984ede [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:49.064726 28296 raft_consensus.cc:504] T 00000000000000000000000000000000 P 792615d15172496882a2f1dc04984ede [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "792615d15172496882a2f1dc04984ede" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 37869 } }
I20260808 18:22:49.065135 28296 leader_election.cc:302] T 00000000000000000000000000000000 P 792615d15172496882a2f1dc04984ede [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 792615d15172496882a2f1dc04984ede; no voters:
I20260808 18:22:49.065464 28296 leader_election.cc:288] T 00000000000000000000000000000000 P 792615d15172496882a2f1dc04984ede [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:22:49.065691 28301 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 792615d15172496882a2f1dc04984ede [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:22:49.065965 28301 raft_consensus.cc:686] T 00000000000000000000000000000000 P 792615d15172496882a2f1dc04984ede [term 1 LEADER]: Becoming Leader. State: Replica: 792615d15172496882a2f1dc04984ede, State: Running, Role: LEADER
I20260808 18:22:49.066246 28301 consensus_queue.cc:237] T 00000000000000000000000000000000 P 792615d15172496882a2f1dc04984ede [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "792615d15172496882a2f1dc04984ede" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 37869 } }
I20260808 18:22:49.066712 28296 sys_catalog.cc:564] T 00000000000000000000000000000000 P 792615d15172496882a2f1dc04984ede [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:22:49.068024 28303 sys_catalog.cc:455] T 00000000000000000000000000000000 P 792615d15172496882a2f1dc04984ede [sys.catalog]: SysCatalogTable state changed. Reason: New leader 792615d15172496882a2f1dc04984ede. Latest consensus state: current_term: 1 leader_uuid: "792615d15172496882a2f1dc04984ede" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "792615d15172496882a2f1dc04984ede" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 37869 } } }
I20260808 18:22:49.068169 28303 sys_catalog.cc:458] T 00000000000000000000000000000000 P 792615d15172496882a2f1dc04984ede [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:49.068547 28311 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:22:49.068497 28302 sys_catalog.cc:455] T 00000000000000000000000000000000 P 792615d15172496882a2f1dc04984ede [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "792615d15172496882a2f1dc04984ede" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "792615d15172496882a2f1dc04984ede" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 37869 } } }
I20260808 18:22:49.068655 28302 sys_catalog.cc:458] T 00000000000000000000000000000000 P 792615d15172496882a2f1dc04984ede [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:49.071750 28311 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:22:49.076948 28311 catalog_manager.cc:1347] Generated new cluster ID: 42b51698b1a8464fa9a30581be8b9f3a
I20260808 18:22:49.077026 28311 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:22:49.086401 28311 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:22:49.087489 28311 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:22:49.094794 28311 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 792615d15172496882a2f1dc04984ede: Generated new TSK 0
I20260808 18:22:49.095510 28311 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.72.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab.
Entry for principal kudu/127.23.72.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab.
I20260808 18:22:49.126479 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.129:0
--local_ip_for_outbound_sockets=127.23.72.129
--webserver_interface=127.23.72.129
--webserver_port=0
--tserver_master_addrs=127.23.72.190:37869
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab
--principal=kudu/127.23.72.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:35927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:49.238292 28325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:49.238575 28325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:49.238655 28325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:49.243538 28325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:49.243623 28325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:49.243717 28325 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.129
I20260808 18:22:49.248955 28325 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:35927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab
--principal=kudu/127.23.72.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.72.129
--webserver_port=0
--tserver_master_addrs=127.23.72.190:37869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28325
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.129
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:49.250020 28325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:49.251034 28325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:49.260288 28333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:49.260337 28330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:49.260488 28325 server_base.cc:1034] running on GCE node
W20260808 18:22:49.260318 28331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:49.260916 28325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:49.261477 28325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:49.262638 28325 hybrid_clock.cc:648] HybridClock initialized: now 1786213369262604 us; error 44 us; skew 500 ppm
Aug 08 18:22:49 dist-test-slave-5k54 krb5kdc[28208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213369, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:49.265455 28325 init.cc:377] Logged in from keytab as kudu/127.23.72.129@KRBTEST.COM (short username kudu)
I20260808 18:22:49.266150 28325 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:49.266242 28325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:49.266476 28325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:49.268615 28325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-0/data/instance:
uuid: "6c5b8e5968864e07be3476cd7f55d677"
format_stamp: "Formatted at 2026-08-08 18:22:49 on dist-test-slave-5k54"
server_key: "fbf21cfd99961fc18d88927f9477eb96"
server_key_iv: "ce5147e9bbdcfe2e4daf48c4f2944380"
server_key_version: "encryptionkey@0"
I20260808 18:22:49.269171 28325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance:
uuid: "6c5b8e5968864e07be3476cd7f55d677"
format_stamp: "Formatted at 2026-08-08 18:22:49 on dist-test-slave-5k54"
server_key: "fbf21cfd99961fc18d88927f9477eb96"
server_key_iv: "ce5147e9bbdcfe2e4daf48c4f2944380"
server_key_version: "encryptionkey@0"
I20260808 18:22:49.272733 28325 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:22:49.275097 28340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:49.276292 28325 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:22:49.276435 28325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-0/wal
uuid: "6c5b8e5968864e07be3476cd7f55d677"
format_stamp: "Formatted at 2026-08-08 18:22:49 on dist-test-slave-5k54"
server_key: "fbf21cfd99961fc18d88927f9477eb96"
server_key_iv: "ce5147e9bbdcfe2e4daf48c4f2944380"
server_key_version: "encryptionkey@0"
I20260808 18:22:49.276546 28325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:49.294862 28325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:49.297627 28325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:49.297839 28325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:49.298425 28325 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:22:49.299214 28325 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:22:49.299288 28325 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:49.299355 28325 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:22:49.299409 28325 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:49.308223 28325 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.129:39301
I20260808 18:22:49.308250 28466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.129:39301 every 8 connection(s)
I20260808 18:22:49.308776 28325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
I20260808 18:22:49.313760 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 28325
I20260808 18:22:49.313894 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance
I20260808 18:22:49.314206 23842 external_mini_cluster.cc:1428] Setting key d1d836d7b3bc35eba7a2b855be5dc1bc
Aug 08 18:22:49 dist-test-slave-5k54 krb5kdc[28208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213369, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:22:49.323776 28299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:49.310517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:39029 (local address 127.23.72.190:37869)
0808 18:22:49.310692 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:49.310696 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:49.311507 (+ 811us) server_negotiation.cc:407] Connection header received
0808 18:22:49.312949 (+ 1442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:49.312952 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:49.312999 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:49.313074 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:49.314501 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:49.315149 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:49.316820 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:49.317067 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:49.317145 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:49.319863 (+ 2718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:49.319867 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:49.319870 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:49.319901 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:49.321464 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:49.322132 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:49.322135 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:49.322136 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:49.322219 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:49.322639 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:49.322641 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:49.322643 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:49.322810 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:49.322883 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:49.323552 (+ 669us) server_negotiation.cc:299] Negotiation successful
0808 18:22:49.323658 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 18:22:49.324980 28470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:49.310855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37869 (local address 127.23.72.129:39029)
0808 18:22:49.311323 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:49.311366 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:49.312643 (+ 1277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:49.313232 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:49.313239 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:49.313668 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:49.314351 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:49.314366 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:49.315277 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:49.315281 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:49.316670 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:49.316684 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:49.317218 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:49.317225 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:49.317470 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:49.318155 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:49.318170 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:49.319688 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:49.321595 (+ 1907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:49.321605 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:49.321622 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:49.322002 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:49.322323 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:49.322330 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:49.322333 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:49.322535 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:49.322917 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:49.322925 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:49.323261 (+ 336us) client_negotiation.cc:769] Sending connection context
0808 18:22:49.323507 (+ 246us) client_negotiation.cc:240] Negotiation successful
0808 18:22:49.323863 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":105,"threads_started":1}
WARNING: no policy specified for kudu/127.23.72.130@KRBTEST.COM; defaulting to no policy
I20260808 18:22:49.326365 28468 heartbeater.cc:347] Connected to a master server at 127.23.72.190:37869
I20260808 18:22:49.326611 28468 heartbeater.cc:464] Registering TS with master...
I20260808 18:22:49.327099 28468 heartbeater.cc:511] Master 127.23.72.190:37869 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.130@KRBTEST.COM" created.
I20260808 18:22:49.328653 28257 ts_manager.cc:194] Registered new tserver with Master: 6c5b8e5968864e07be3476cd7f55d677 (127.23.72.129:39301)
I20260808 18:22:49.329767 28257 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.129@KRBTEST.COM'} at 127.23.72.129:39029
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab.
Entry for principal kudu/127.23.72.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab.
I20260808 18:22:49.346464 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.130:0
--local_ip_for_outbound_sockets=127.23.72.130
--webserver_interface=127.23.72.130
--webserver_port=0
--tserver_master_addrs=127.23.72.190:37869
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab
--principal=kudu/127.23.72.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:35927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:49.455873 28473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:49.456115 28473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:49.456158 28473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:49.460083 28473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:49.460130 28473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:49.460186 28473 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.130
I20260808 18:22:49.465251 28473 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:35927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab
--principal=kudu/127.23.72.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.72.130
--webserver_port=0
--tserver_master_addrs=127.23.72.190:37869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28473
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.130
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:49.466159 28473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:49.467144 28473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:49.476210 28481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:49.476176 28478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:49.476400 28473 server_base.cc:1034] running on GCE node
W20260808 18:22:49.476181 28479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:49.476913 28473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:49.477504 28473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:49.478667 28473 hybrid_clock.cc:648] HybridClock initialized: now 1786213369478652 us; error 25 us; skew 500 ppm
Aug 08 18:22:49 dist-test-slave-5k54 krb5kdc[28208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213369, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:49.481659 28473 init.cc:377] Logged in from keytab as kudu/127.23.72.130@KRBTEST.COM (short username kudu)
I20260808 18:22:49.482409 28473 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:49.482499 28473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:49.482730 28473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:49.484736 28473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-1/data/instance:
uuid: "768b4c9b77244208ba40dde9a269778b"
format_stamp: "Formatted at 2026-08-08 18:22:49 on dist-test-slave-5k54"
server_key: "753d1df2fc4bb9b42e603568e9c776cc"
server_key_iv: "2281f133a4daf5ba2fb32ea1f1486002"
server_key_version: "encryptionkey@0"
I20260808 18:22:49.485317 28473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance:
uuid: "768b4c9b77244208ba40dde9a269778b"
format_stamp: "Formatted at 2026-08-08 18:22:49 on dist-test-slave-5k54"
server_key: "753d1df2fc4bb9b42e603568e9c776cc"
server_key_iv: "2281f133a4daf5ba2fb32ea1f1486002"
server_key_version: "encryptionkey@0"
I20260808 18:22:49.489144 28473 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260808 18:22:49.491402 28488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:49.492684 28473 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 18:22:49.492815 28473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-1/wal
uuid: "768b4c9b77244208ba40dde9a269778b"
format_stamp: "Formatted at 2026-08-08 18:22:49 on dist-test-slave-5k54"
server_key: "753d1df2fc4bb9b42e603568e9c776cc"
server_key_iv: "2281f133a4daf5ba2fb32ea1f1486002"
server_key_version: "encryptionkey@0"
I20260808 18:22:49.492938 28473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:49.504242 28473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:49.507899 28473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:49.508090 28473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:49.508644 28473 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:22:49.509444 28473 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:22:49.509514 28473 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:49.509583 28473 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:22:49.509634 28473 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:49.518719 28473 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.130:44627
I20260808 18:22:49.518766 28609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.130:44627 every 8 connection(s)
I20260808 18:22:49.519326 28473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
I20260808 18:22:49.522840 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 28473
I20260808 18:22:49.522998 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance
I20260808 18:22:49.523370 23842 external_mini_cluster.cc:1428] Setting key 5f1737d8d661939e044a1f42c3ed5ce6
Aug 08 18:22:49 dist-test-slave-5k54 krb5kdc[28208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213369, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.130@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.72.131@KRBTEST.COM; defaulting to no policy
I20260808 18:22:49.534528 28299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:49.521386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:40555 (local address 127.23.72.190:37869)
0808 18:22:49.521540 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:49.521544 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:49.522358 (+ 814us) server_negotiation.cc:407] Connection header received
0808 18:22:49.524025 (+ 1667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:49.524028 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:49.524089 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:49.524202 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:49.525649 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:49.526203 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:49.527704 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:49.528091 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:49.528214 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:49.530668 (+ 2454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:49.530675 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:49.530680 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:49.530715 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:49.532408 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:49.532923 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:49.532927 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:49.532929 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:49.533022 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:49.533404 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:49.533411 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:49.533414 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:49.533674 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:49.533776 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:49.534270 (+ 494us) server_negotiation.cc:299] Negotiation successful
0808 18:22:49.534408 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 18:22:49.535346 28612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:49.521740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37869 (local address 127.23.72.130:40555)
0808 18:22:49.522191 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:49.522234 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:49.523807 (+ 1573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:49.524380 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:49.524388 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:49.524801 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:49.525487 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:49.525509 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:49.526335 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:49.526342 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:49.527578 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:49.527588 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:49.528245 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:49.528247 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:49.528390 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:49.528971 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:49.528987 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:49.530500 (+ 1513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:49.532553 (+ 2053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:49.532557 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:49.532568 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:49.532816 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:49.533125 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:49.533129 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:49.533131 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:49.533298 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:49.533776 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:49.533781 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:49.534022 (+ 241us) client_negotiation.cc:769] Sending connection context
0808 18:22:49.534185 (+ 163us) client_negotiation.cc:240] Negotiation successful
0808 18:22:49.534449 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":104,"threads_started":1}
I20260808 18:22:49.536545 28610 heartbeater.cc:347] Connected to a master server at 127.23.72.190:37869
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.131@KRBTEST.COM" created.
I20260808 18:22:49.536821 28610 heartbeater.cc:464] Registering TS with master...
I20260808 18:22:49.537339 28610 heartbeater.cc:511] Master 127.23.72.190:37869 requested a full tablet report, sending...
I20260808 18:22:49.538463 28257 ts_manager.cc:194] Registered new tserver with Master: 768b4c9b77244208ba40dde9a269778b (127.23.72.130:44627)
I20260808 18:22:49.539054 28257 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.130@KRBTEST.COM'} at 127.23.72.130:40555
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab.
Entry for principal kudu/127.23.72.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab.
I20260808 18:22:49.554183 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.131:0
--local_ip_for_outbound_sockets=127.23.72.131
--webserver_interface=127.23.72.131
--webserver_port=0
--tserver_master_addrs=127.23.72.190:37869
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab
--principal=kudu/127.23.72.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:35927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:49.665686 28616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:49.665932 28616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:49.665973 28616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:49.670132 28616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:49.670187 28616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:49.670277 28616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.131
I20260808 18:22:49.675338 28616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:35927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab
--principal=kudu/127.23.72.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.72.131
--webserver_port=0
--tserver_master_addrs=127.23.72.190:37869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.131
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:49.676347 28616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:49.677385 28616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:49.686676 28622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:49.686691 28621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:49.686728 28624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:49.686892 28616 server_base.cc:1034] running on GCE node
I20260808 18:22:49.687528 28616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:49.688135 28616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:49.689332 28616 hybrid_clock.cc:648] HybridClock initialized: now 1786213369689308 us; error 32 us; skew 500 ppm
Aug 08 18:22:49 dist-test-slave-5k54 krb5kdc[28208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213369, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:49.692663 28616 init.cc:377] Logged in from keytab as kudu/127.23.72.131@KRBTEST.COM (short username kudu)
I20260808 18:22:49.693389 28616 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:49.693486 28616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:49.693708 28616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:49.695763 28616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-2/data/instance:
uuid: "2379ededc56c48c48fef3ee5d7fd4dcc"
format_stamp: "Formatted at 2026-08-08 18:22:49 on dist-test-slave-5k54"
server_key: "6abd9ce097083c9b7c940e69192b7c47"
server_key_iv: "18956f0a0ea373dc24ef750c5698132e"
server_key_version: "encryptionkey@0"
I20260808 18:22:49.696329 28616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance:
uuid: "2379ededc56c48c48fef3ee5d7fd4dcc"
format_stamp: "Formatted at 2026-08-08 18:22:49 on dist-test-slave-5k54"
server_key: "6abd9ce097083c9b7c940e69192b7c47"
server_key_iv: "18956f0a0ea373dc24ef750c5698132e"
server_key_version: "encryptionkey@0"
I20260808 18:22:49.699863 28616 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:22:49.702121 28630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:49.703258 28616 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:22:49.703390 28616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-2/wal
uuid: "2379ededc56c48c48fef3ee5d7fd4dcc"
format_stamp: "Formatted at 2026-08-08 18:22:49 on dist-test-slave-5k54"
server_key: "6abd9ce097083c9b7c940e69192b7c47"
server_key_iv: "18956f0a0ea373dc24ef750c5698132e"
server_key_version: "encryptionkey@0"
I20260808 18:22:49.703562 28616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:49.720126 28616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:49.722952 28616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:49.723167 28616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:49.723812 28616 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:22:49.724623 28616 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:22:49.724695 28616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:49.724781 28616 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:22:49.724829 28616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:49.733579 28616 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.131:32869
I20260808 18:22:49.733598 28754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.131:32869 every 8 connection(s)
I20260808 18:22:49.734153 28616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
I20260808 18:22:49.741022 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 28616
I20260808 18:22:49.741142 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance
I20260808 18:22:49.741504 23842 external_mini_cluster.cc:1428] Setting key 4097b6cabd2216b156be24433301566d
Aug 08 18:22:49 dist-test-slave-5k54 krb5kdc[28208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213369, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.131@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:22:49.746874 28299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:49.735833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:58693 (local address 127.23.72.190:37869)
0808 18:22:49.736006 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:49.736011 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:49.736797 (+ 786us) server_negotiation.cc:407] Connection header received
0808 18:22:49.738012 (+ 1215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:49.738015 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:49.738061 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:49.738134 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:49.739221 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:49.739808 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:49.740888 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:49.741249 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:49.741368 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:49.743886 (+ 2518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:49.743889 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:49.743891 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:49.743931 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:49.745164 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:49.745615 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:49.745617 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:49.745618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:49.745692 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:49.745987 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:49.745989 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:49.745990 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:49.746156 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:49.746230 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:49.746658 (+ 428us) server_negotiation.cc:299] Negotiation successful
0808 18:22:49.746757 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 18:22:49.747725 28758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:49.736209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37869 (local address 127.23.72.131:58693)
0808 18:22:49.736651 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:49.736694 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:49.737826 (+ 1132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:49.738292 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:49.738298 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:49.738648 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:49.739101 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:49.739113 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:49.739918 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:49.739920 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:49.740772 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:49.740781 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:49.741402 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:49.741405 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:49.741538 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:49.742221 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:49.742237 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:49.743745 (+ 1508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:49.745264 (+ 1519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:49.745269 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:49.745280 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:49.745529 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:49.745770 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:49.745773 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:49.745774 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:49.745908 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:49.746242 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:49.746247 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:49.746459 (+ 212us) client_negotiation.cc:769] Sending connection context
0808 18:22:49.746607 (+ 148us) client_negotiation.cc:240] Negotiation successful
0808 18:22:49.746862 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":116,"threads_started":1}
I20260808 18:22:49.748793 28755 heartbeater.cc:347] Connected to a master server at 127.23.72.190:37869
I20260808 18:22:49.748991 28755 heartbeater.cc:464] Registering TS with master...
I20260808 18:22:49.749375 28755 heartbeater.cc:511] Master 127.23.72.190:37869 requested a full tablet report, sending...
I20260808 18:22:49.750267 28257 ts_manager.cc:194] Registered new tserver with Master: 2379ededc56c48c48fef3ee5d7fd4dcc (127.23.72.131:32869)
I20260808 18:22:49.750819 28257 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.131@KRBTEST.COM'} at 127.23.72.131:58693
I20260808 18:22:49.755120 23842 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:22:49.764261 28299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:49.757065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40826 (local address 127.23.72.190:37869)
0808 18:22:49.757209 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:49.757214 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:49.757421 (+ 207us) server_negotiation.cc:407] Connection header received
0808 18:22:49.757570 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:49.757573 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:49.757613 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:49.757701 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:49.758372 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:49.758874 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:49.759774 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:49.760012 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:49.760117 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:49.761351 (+ 1234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:49.761354 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:49.761357 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:49.761381 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:49.762544 (+ 1163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:49.762971 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:49.762975 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:49.762977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:49.763061 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:49.763334 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:49.763338 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:49.763340 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:49.763616 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:49.763754 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:49.763993 (+ 239us) server_negotiation.cc:299] Negotiation successful
0808 18:22:49.764109 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 18:22:49.766893 28257 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40826:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:22:49.770179 28257 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:22:49.783800 28772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:49.778776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59794 (local address 127.23.72.131:32869)
0808 18:22:49.779190 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:49.779205 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:49.779228 (+ 23us) server_negotiation.cc:407] Connection header received
0808 18:22:49.779301 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:49.779306 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:49.779547 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:49.779698 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:22:49.780199 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:49.781113 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:49.782258 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:49.782862 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:49.782990 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:49.783135 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:49.783513 (+ 378us) server_negotiation.cc:299] Negotiation successful
0808 18:22:49.783627 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":118,"threads_started":1}
I20260808 18:22:49.784186 28768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:49.778373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.131:32869 (local address 127.0.0.1:59794)
0808 18:22:49.778966 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:49.779015 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:49.779144 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:49.779898 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:49.779904 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:49.779927 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:22:49.780053 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:49.780062 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:49.781229 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:49.781235 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:49.782111 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:49.782146 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:49.782991 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:49.782994 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:49.783198 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:49.783254 (+ 56us) client_negotiation.cc:769] Sending connection context
0808 18:22:49.783361 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 18:22:49.784049 (+ 688us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"mutex_wait_us":53,"thread_start_us":118,"threads_started":1}
I20260808 18:22:49.785072 28773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:49.778977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.130:44627 (local address 127.0.0.1:57770)
0808 18:22:49.780098 (+ 1121us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:49.780113 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:49.780191 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:49.780778 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:49.780781 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:49.780799 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:22:49.780905 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:49.780910 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:49.782185 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:49.782191 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:49.783482 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:49.783494 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:49.784666 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:49.784670 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:49.784779 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:49.784802 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 18:22:49.784880 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 18:22:49.784946 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1059,"mutex_wait_us":77,"thread_start_us":95,"threads_started":1}
I20260808 18:22:49.785172 28774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:49.779691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57770 (local address 127.23.72.130:44627)
0808 18:22:49.780332 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:49.780343 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:49.780364 (+ 21us) server_negotiation.cc:407] Connection header received
0808 18:22:49.780448 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:49.780456 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:49.780621 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:49.780763 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:22:49.781019 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:49.782043 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:49.783625 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:49.784547 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:49.784654 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:49.784803 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:49.784920 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 18:22:49.785017 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":131,"threads_started":1}
I20260808 18:22:49.787070 28683 tablet_service.cc:1467] Processing CreateTablet for tablet 55b9a7e9522e43359e6538d69adf909d (DEFAULT_TABLE table=test-table [id=9f1e16daa2784bda95fdc5e34ba1918b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:22:49.787730 28541 tablet_service.cc:1467] Processing CreateTablet for tablet 55b9a7e9522e43359e6538d69adf909d (DEFAULT_TABLE table=test-table [id=9f1e16daa2784bda95fdc5e34ba1918b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:22:49.788067 28683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55b9a7e9522e43359e6538d69adf909d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:49.788597 28541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55b9a7e9522e43359e6538d69adf909d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:49.794281 28769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:49.778287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:39301 (local address 127.0.0.1:45824)
0808 18:22:49.779078 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:49.779094 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:49.779178 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:49.780001 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:49.780003 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:49.780021 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:22:49.780128 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:49.780137 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:49.781586 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:49.781589 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:49.783584 (+ 1995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:49.783594 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:49.785042 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:49.785045 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:49.793951 (+ 8906us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:49.793979 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 18:22:49.794065 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 18:22:49.794143 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"mutex_wait_us":167,"thread_start_us":69,"threads_started":1}
I20260808 18:22:49.794823 28776 tablet_bootstrap.cc:492] T 55b9a7e9522e43359e6538d69adf909d P 2379ededc56c48c48fef3ee5d7fd4dcc: Bootstrap starting.
I20260808 18:22:49.795076 28770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:49.778429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45824 (local address 127.23.72.129:39301)
0808 18:22:49.779416 (+ 987us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:49.779465 (+ 49us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:49.779488 (+ 23us) server_negotiation.cc:407] Connection header received
0808 18:22:49.779566 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:49.779571 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:49.779843 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:49.780006 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:22:49.780268 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:49.781440 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:49.783705 (+ 2265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:49.784721 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:49.784848 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:49.784979 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:49.794793 (+ 9814us) server_negotiation.cc:299] Negotiation successful
0808 18:22:49.794918 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":894,"thread_start_us":72,"threads_started":1}
I20260808 18:22:49.795401 28778 tablet_bootstrap.cc:492] T 55b9a7e9522e43359e6538d69adf909d P 768b4c9b77244208ba40dde9a269778b: Bootstrap starting.
I20260808 18:22:49.797237 28776 tablet_bootstrap.cc:654] T 55b9a7e9522e43359e6538d69adf909d P 2379ededc56c48c48fef3ee5d7fd4dcc: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:49.797339 28778 tablet_bootstrap.cc:654] T 55b9a7e9522e43359e6538d69adf909d P 768b4c9b77244208ba40dde9a269778b: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:49.798328 28778 log.cc:824] T 55b9a7e9522e43359e6538d69adf909d P 768b4c9b77244208ba40dde9a269778b: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:49.797971 28392 tablet_service.cc:1467] Processing CreateTablet for tablet 55b9a7e9522e43359e6538d69adf909d (DEFAULT_TABLE table=test-table [id=9f1e16daa2784bda95fdc5e34ba1918b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:22:49.798429 28776 log.cc:824] T 55b9a7e9522e43359e6538d69adf909d P 2379ededc56c48c48fef3ee5d7fd4dcc: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:49.798969 28392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55b9a7e9522e43359e6538d69adf909d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:49.800427 28778 tablet_bootstrap.cc:492] T 55b9a7e9522e43359e6538d69adf909d P 768b4c9b77244208ba40dde9a269778b: No bootstrap required, opened a new log
I20260808 18:22:49.800609 28778 ts_tablet_manager.cc:1397] T 55b9a7e9522e43359e6538d69adf909d P 768b4c9b77244208ba40dde9a269778b: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260808 18:22:49.800625 28776 tablet_bootstrap.cc:492] T 55b9a7e9522e43359e6538d69adf909d P 2379ededc56c48c48fef3ee5d7fd4dcc: No bootstrap required, opened a new log
I20260808 18:22:49.800805 28776 ts_tablet_manager.cc:1397] T 55b9a7e9522e43359e6538d69adf909d P 2379ededc56c48c48fef3ee5d7fd4dcc: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 18:22:49.803403 28776 raft_consensus.cc:348] T 55b9a7e9522e43359e6538d69adf909d P 2379ededc56c48c48fef3ee5d7fd4dcc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c5b8e5968864e07be3476cd7f55d677" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 39301 } } peers { permanent_uuid: "2379ededc56c48c48fef3ee5d7fd4dcc" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 32869 } } peers { permanent_uuid: "768b4c9b77244208ba40dde9a269778b" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 44627 } }
I20260808 18:22:49.803701 28776 raft_consensus.cc:374] T 55b9a7e9522e43359e6538d69adf909d P 2379ededc56c48c48fef3ee5d7fd4dcc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:49.803755 28776 raft_consensus.cc:729] T 55b9a7e9522e43359e6538d69adf909d P 2379ededc56c48c48fef3ee5d7fd4dcc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2379ededc56c48c48fef3ee5d7fd4dcc, State: Initialized, Role: FOLLOWER
I20260808 18:22:49.804195 28776 consensus_queue.cc:260] T 55b9a7e9522e43359e6538d69adf909d P 2379ededc56c48c48fef3ee5d7fd4dcc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c5b8e5968864e07be3476cd7f55d677" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 39301 } } peers { permanent_uuid: "2379ededc56c48c48fef3ee5d7fd4dcc" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 32869 } } peers { permanent_uuid: "768b4c9b77244208ba40dde9a269778b" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 44627 } }
I20260808 18:22:49.804418 28778 raft_consensus.cc:348] T 55b9a7e9522e43359e6538d69adf909d P 768b4c9b77244208ba40dde9a269778b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c5b8e5968864e07be3476cd7f55d677" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 39301 } } peers { permanent_uuid: "2379ededc56c48c48fef3ee5d7fd4dcc" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 32869 } } peers { permanent_uuid: "768b4c9b77244208ba40dde9a269778b" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 44627 } }
I20260808 18:22:49.804695 28778 raft_consensus.cc:374] T 55b9a7e9522e43359e6538d69adf909d P 768b4c9b77244208ba40dde9a269778b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:49.804778 28778 raft_consensus.cc:729] T 55b9a7e9522e43359e6538d69adf909d P 768b4c9b77244208ba40dde9a269778b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 768b4c9b77244208ba40dde9a269778b, State: Initialized, Role: FOLLOWER
I20260808 18:22:49.804919 28755 heartbeater.cc:503] Master 127.23.72.190:37869 was elected leader, sending a full tablet report...
I20260808 18:22:49.805191 28776 ts_tablet_manager.cc:1428] T 55b9a7e9522e43359e6538d69adf909d P 2379ededc56c48c48fef3ee5d7fd4dcc: Time spent starting tablet: real 0.004s user 0.000s sys 0.005s
I20260808 18:22:49.805212 28778 consensus_queue.cc:260] T 55b9a7e9522e43359e6538d69adf909d P 768b4c9b77244208ba40dde9a269778b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c5b8e5968864e07be3476cd7f55d677" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 39301 } } peers { permanent_uuid: "2379ededc56c48c48fef3ee5d7fd4dcc" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 32869 } } peers { permanent_uuid: "768b4c9b77244208ba40dde9a269778b" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 44627 } }
I20260808 18:22:49.805996 28783 tablet_bootstrap.cc:492] T 55b9a7e9522e43359e6538d69adf909d P 6c5b8e5968864e07be3476cd7f55d677: Bootstrap starting.
I20260808 18:22:49.806305 28778 ts_tablet_manager.cc:1428] T 55b9a7e9522e43359e6538d69adf909d P 768b4c9b77244208ba40dde9a269778b: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 18:22:49.806627 28610 heartbeater.cc:503] Master 127.23.72.190:37869 was elected leader, sending a full tablet report...
I20260808 18:22:49.808379 28783 tablet_bootstrap.cc:654] T 55b9a7e9522e43359e6538d69adf909d P 6c5b8e5968864e07be3476cd7f55d677: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:49.809648 28783 log.cc:824] T 55b9a7e9522e43359e6538d69adf909d P 6c5b8e5968864e07be3476cd7f55d677: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:49.811476 28783 tablet_bootstrap.cc:492] T 55b9a7e9522e43359e6538d69adf909d P 6c5b8e5968864e07be3476cd7f55d677: No bootstrap required, opened a new log
I20260808 18:22:49.811663 28783 ts_tablet_manager.cc:1397] T 55b9a7e9522e43359e6538d69adf909d P 6c5b8e5968864e07be3476cd7f55d677: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 18:22:49.815954 28783 raft_consensus.cc:348] T 55b9a7e9522e43359e6538d69adf909d P 6c5b8e5968864e07be3476cd7f55d677 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c5b8e5968864e07be3476cd7f55d677" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 39301 } } peers { permanent_uuid: "2379ededc56c48c48fef3ee5d7fd4dcc" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 32869 } } peers { permanent_uuid: "768b4c9b77244208ba40dde9a269778b" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 44627 } }
I20260808 18:22:49.816473 28783 raft_consensus.cc:374] T 55b9a7e9522e43359e6538d69adf909d P 6c5b8e5968864e07be3476cd7f55d677 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:49.816677 28783 raft_consensus.cc:729] T 55b9a7e9522e43359e6538d69adf909d P 6c5b8e5968864e07be3476cd7f55d677 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c5b8e5968864e07be3476cd7f55d677, State: Initialized, Role: FOLLOWER
I20260808 18:22:49.817241 28783 consensus_queue.cc:260] T 55b9a7e9522e43359e6538d69adf909d P 6c5b8e5968864e07be3476cd7f55d677 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c5b8e5968864e07be3476cd7f55d677" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 39301 } } peers { permanent_uuid: "2379ededc56c48c48fef3ee5d7fd4dcc" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 32869 } } peers { permanent_uuid: "768b4c9b77244208ba40dde9a269778b" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 44627 } }
I20260808 18:22:49.817929 28468 heartbeater.cc:503] Master 127.23.72.190:37869 was elected leader, sending a full tablet report...
I20260808 18:22:49.818203 28783 ts_tablet_manager.cc:1428] T 55b9a7e9522e43359e6538d69adf909d P 6c5b8e5968864e07be3476cd7f55d677: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 18:22:49.853628 28785 raft_consensus.cc:482] T 55b9a7e9522e43359e6538d69adf909d P 6c5b8e5968864e07be3476cd7f55d677 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:22:49.853839 28785 raft_consensus.cc:504] T 55b9a7e9522e43359e6538d69adf909d P 6c5b8e5968864e07be3476cd7f55d677 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c5b8e5968864e07be3476cd7f55d677" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 39301 } } peers { permanent_uuid: "2379ededc56c48c48fef3ee5d7fd4dcc" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 32869 } } peers { permanent_uuid: "768b4c9b77244208ba40dde9a269778b" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 44627 } }
I20260808 18:22:49.855013 28785 leader_election.cc:288] T 55b9a7e9522e43359e6538d69adf909d P 6c5b8e5968864e07be3476cd7f55d677 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2379ededc56c48c48fef3ee5d7fd4dcc (127.23.72.131:32869), 768b4c9b77244208ba40dde9a269778b (127.23.72.130:44627)
I20260808 18:22:49.859442 28788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:49.855315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.130:44627 (local address 127.23.72.129:60757)
0808 18:22:49.856045 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:49.856060 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:49.856185 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:49.856465 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:49.856468 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:49.856485 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:22:49.856586 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:49.856592 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:49.857439 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:49.857441 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:49.858263 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:49.858271 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:49.858944 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:49.858947 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:49.859066 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:49.859084 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 18:22:49.859169 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 18:22:49.859277 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650,"mutex_wait_us":24,"thread_start_us":484,"threads_started":1}
I20260808 18:22:49.859408 28772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:49.855361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:41101 (local address 127.23.72.131:32869)
0808 18:22:49.855543 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:49.855550 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:49.855871 (+ 321us) server_negotiation.cc:407] Connection header received
0808 18:22:49.855935 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:49.855938 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:49.855990 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:49.856076 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:22:49.856434 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:49.857074 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:49.858437 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:49.858811 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:49.858942 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:49.858996 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:49.859188 (+ 192us) server_negotiation.cc:299] Negotiation successful
0808 18:22:49.859263 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260808 18:22:49.859411 28774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:49.855687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:60757 (local address 127.23.72.130:44627)
0808 18:22:49.855829 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:49.855835 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:49.856121 (+ 286us) server_negotiation.cc:407] Connection header received
0808 18:22:49.856292 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:49.856295 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:49.856371 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:49.856454 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:22:49.856726 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:49.857331 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:49.858393 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:49.858809 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:49.858932 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:49.858984 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:49.859191 (+ 207us) server_negotiation.cc:299] Negotiation successful
0808 18:22:49.859283 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 18:22:49.859500 28787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:49.855251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.131:32869 (local address 127.23.72.129:41101)
0808 18:22:49.855648 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:49.855664 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:49.855806 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:49.856116 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:49.856123 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:49.856150 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:22:49.856312 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:49.856319 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:49.857224 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:49.857229 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:49.858283 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:49.858294 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:49.858931 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:49.858934 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:49.859049 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:49.859074 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 18:22:49.859171 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 18:22:49.859313 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"mutex_wait_us":105,"spinlock_wait_cycles":7936,"thread_start_us":113,"threads_started":1}
I20260808 18:22:49.860118 28563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55b9a7e9522e43359e6538d69adf909d" candidate_uuid: "6c5b8e5968864e07be3476cd7f55d677" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "768b4c9b77244208ba40dde9a269778b" is_pre_election: true
I20260808 18:22:49.860286 28705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55b9a7e9522e43359e6538d69adf909d" candidate_uuid: "6c5b8e5968864e07be3476cd7f55d677" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2379ededc56c48c48fef3ee5d7fd4dcc" is_pre_election: true
I20260808 18:22:49.860477 28563 raft_consensus.cc:2454] T 55b9a7e9522e43359e6538d69adf909d P 768b4c9b77244208ba40dde9a269778b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c5b8e5968864e07be3476cd7f55d677 in term 0.
I20260808 18:22:49.860527 28705 raft_consensus.cc:2454] T 55b9a7e9522e43359e6538d69adf909d P 2379ededc56c48c48fef3ee5d7fd4dcc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c5b8e5968864e07be3476cd7f55d677 in term 0.
I20260808 18:22:49.860996 28342 leader_election.cc:302] T 55b9a7e9522e43359e6538d69adf909d P 6c5b8e5968864e07be3476cd7f55d677 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2379ededc56c48c48fef3ee5d7fd4dcc, 6c5b8e5968864e07be3476cd7f55d677; no voters:
I20260808 18:22:49.861233 28785 raft_consensus.cc:2781] T 55b9a7e9522e43359e6538d69adf909d P 6c5b8e5968864e07be3476cd7f55d677 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:22:49.861332 28785 raft_consensus.cc:482] T 55b9a7e9522e43359e6538d69adf909d P 6c5b8e5968864e07be3476cd7f55d677 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:22:49.861392 28785 raft_consensus.cc:3037] T 55b9a7e9522e43359e6538d69adf909d P 6c5b8e5968864e07be3476cd7f55d677 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:49.862879 28785 raft_consensus.cc:504] T 55b9a7e9522e43359e6538d69adf909d P 6c5b8e5968864e07be3476cd7f55d677 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c5b8e5968864e07be3476cd7f55d677" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 39301 } } peers { permanent_uuid: "2379ededc56c48c48fef3ee5d7fd4dcc" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 32869 } } peers { permanent_uuid: "768b4c9b77244208ba40dde9a269778b" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 44627 } }
I20260808 18:22:49.863237 28785 leader_election.cc:288] T 55b9a7e9522e43359e6538d69adf909d P 6c5b8e5968864e07be3476cd7f55d677 [CANDIDATE]: Term 1 election: Requested vote from peers 2379ededc56c48c48fef3ee5d7fd4dcc (127.23.72.131:32869), 768b4c9b77244208ba40dde9a269778b (127.23.72.130:44627)
I20260808 18:22:49.863619 28563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55b9a7e9522e43359e6538d69adf909d" candidate_uuid: "6c5b8e5968864e07be3476cd7f55d677" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "768b4c9b77244208ba40dde9a269778b"
I20260808 18:22:49.863619 28705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55b9a7e9522e43359e6538d69adf909d" candidate_uuid: "6c5b8e5968864e07be3476cd7f55d677" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2379ededc56c48c48fef3ee5d7fd4dcc"
I20260808 18:22:49.863758 28705 raft_consensus.cc:3037] T 55b9a7e9522e43359e6538d69adf909d P 2379ededc56c48c48fef3ee5d7fd4dcc [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:49.863767 28563 raft_consensus.cc:3037] T 55b9a7e9522e43359e6538d69adf909d P 768b4c9b77244208ba40dde9a269778b [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:49.865264 28563 raft_consensus.cc:2454] T 55b9a7e9522e43359e6538d69adf909d P 768b4c9b77244208ba40dde9a269778b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c5b8e5968864e07be3476cd7f55d677 in term 1.
I20260808 18:22:49.865264 28705 raft_consensus.cc:2454] T 55b9a7e9522e43359e6538d69adf909d P 2379ededc56c48c48fef3ee5d7fd4dcc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c5b8e5968864e07be3476cd7f55d677 in term 1.
I20260808 18:22:49.865571 28342 leader_election.cc:302] T 55b9a7e9522e43359e6538d69adf909d P 6c5b8e5968864e07be3476cd7f55d677 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2379ededc56c48c48fef3ee5d7fd4dcc, 6c5b8e5968864e07be3476cd7f55d677; no voters:
I20260808 18:22:49.865753 28785 raft_consensus.cc:2781] T 55b9a7e9522e43359e6538d69adf909d P 6c5b8e5968864e07be3476cd7f55d677 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:22:49.865994 28785 raft_consensus.cc:686] T 55b9a7e9522e43359e6538d69adf909d P 6c5b8e5968864e07be3476cd7f55d677 [term 1 LEADER]: Becoming Leader. State: Replica: 6c5b8e5968864e07be3476cd7f55d677, State: Running, Role: LEADER
I20260808 18:22:49.866340 28785 consensus_queue.cc:237] T 55b9a7e9522e43359e6538d69adf909d P 6c5b8e5968864e07be3476cd7f55d677 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c5b8e5968864e07be3476cd7f55d677" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 39301 } } peers { permanent_uuid: "2379ededc56c48c48fef3ee5d7fd4dcc" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 32869 } } peers { permanent_uuid: "768b4c9b77244208ba40dde9a269778b" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 44627 } }
I20260808 18:22:49.869542 28257 catalog_manager.cc:5495] T 55b9a7e9522e43359e6538d69adf909d P 6c5b8e5968864e07be3476cd7f55d677 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c5b8e5968864e07be3476cd7f55d677 (127.23.72.129). New cstate: current_term: 1 leader_uuid: "6c5b8e5968864e07be3476cd7f55d677" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c5b8e5968864e07be3476cd7f55d677" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 39301 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2379ededc56c48c48fef3ee5d7fd4dcc" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 32869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "768b4c9b77244208ba40dde9a269778b" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 44627 } health_report { overall_health: UNKNOWN } } }
I20260808 18:22:49.883328 28770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:49.879820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45828 (local address 127.23.72.129:39301)
0808 18:22:49.879970 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:49.879975 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:49.879992 (+ 17us) server_negotiation.cc:407] Connection header received
0808 18:22:49.880121 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:49.880124 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:49.880178 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:49.880271 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:22:49.880519 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:49.881084 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:49.881902 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:49.882128 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:49.882213 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:49.882462 (+ 249us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:22:49.882897 (+ 435us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:22:49.882993 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:49.883104 (+ 111us) server_negotiation.cc:299] Negotiation successful
0808 18:22:49.883156 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 18:22:49.889025 28563 raft_consensus.cc:1261] T 55b9a7e9522e43359e6538d69adf909d P 768b4c9b77244208ba40dde9a269778b [term 1 FOLLOWER]: Refusing update from remote peer 6c5b8e5968864e07be3476cd7f55d677: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:22:49.889025 28705 raft_consensus.cc:1261] T 55b9a7e9522e43359e6538d69adf909d P 2379ededc56c48c48fef3ee5d7fd4dcc [term 1 FOLLOWER]: Refusing update from remote peer 6c5b8e5968864e07be3476cd7f55d677: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:22:49.889717 28789 consensus_queue.cc:1035] T 55b9a7e9522e43359e6538d69adf909d P 6c5b8e5968864e07be3476cd7f55d677 [LEADER]: Connected to new peer: Peer: permanent_uuid: "768b4c9b77244208ba40dde9a269778b" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 44627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:22:49.889950 28785 consensus_queue.cc:1035] T 55b9a7e9522e43359e6538d69adf909d P 6c5b8e5968864e07be3476cd7f55d677 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2379ededc56c48c48fef3ee5d7fd4dcc" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 32869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:22:49.897068 28792 mvcc.cc:205] Tried to move back new op lower bound from 7316329963058642944 to 7316329962975174656. Current Snapshot: MvccSnapshot[applied={T|T < 7316329963058642944}]
I20260808 18:22:49.899350 28794 mvcc.cc:205] Tried to move back new op lower bound from 7316329963058642944 to 7316329962975174656. Current Snapshot: MvccSnapshot[applied={T|T < 7316329963058642944}]
I20260808 18:22:49.900285 28793 mvcc.cc:205] Tried to move back new op lower bound from 7316329963058642944 to 7316329962975174656. Current Snapshot: MvccSnapshot[applied={T|T < 7316329963058642944 or (T in {7316329963058642944})}]
I20260808 18:22:49.905469 28256 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40826:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:22:49.905645 28256 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40826:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:22:49.908084 28256 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 792615d15172496882a2f1dc04984ede: Sending DeleteTablet for 3 replicas of tablet 55b9a7e9522e43359e6538d69adf909d
I20260808 18:22:49.908864 28541 tablet_service.cc:1514] Processing DeleteTablet for tablet 55b9a7e9522e43359e6538d69adf909d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:22:49 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:57770
I20260808 18:22:49.908949 28392 tablet_service.cc:1514] Processing DeleteTablet for tablet 55b9a7e9522e43359e6538d69adf909d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:22:49 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:45824
I20260808 18:22:49.908912 28683 tablet_service.cc:1514] Processing DeleteTablet for tablet 55b9a7e9522e43359e6538d69adf909d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:22:49 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:59794
I20260808 18:22:49.909391 28805 tablet_replica.cc:330] T 55b9a7e9522e43359e6538d69adf909d P 2379ededc56c48c48fef3ee5d7fd4dcc: stopping tablet replica
I20260808 18:22:49.909716 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 28325
I20260808 18:22:49.909724 28805 raft_consensus.cc:2229] T 55b9a7e9522e43359e6538d69adf909d P 2379ededc56c48c48fef3ee5d7fd4dcc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:22:49.910025 28805 raft_consensus.cc:2258] T 55b9a7e9522e43359e6538d69adf909d P 2379ededc56c48c48fef3ee5d7fd4dcc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:22:49.912376 28805 ts_tablet_manager.cc:1905] T 55b9a7e9522e43359e6538d69adf909d P 2379ededc56c48c48fef3ee5d7fd4dcc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:22:49.914793 28805 ts_tablet_manager.cc:1918] T 55b9a7e9522e43359e6538d69adf909d P 2379ededc56c48c48fef3ee5d7fd4dcc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:22:49.914920 28805 log.cc:1196] T 55b9a7e9522e43359e6538d69adf909d P 2379ededc56c48c48fef3ee5d7fd4dcc: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213345431840-23842-0/minicluster-data/ts-2/wal/wals/55b9a7e9522e43359e6538d69adf909d
I20260808 18:22:49.915278 28805 ts_tablet_manager.cc:1939] T 55b9a7e9522e43359e6538d69adf909d P 2379ededc56c48c48fef3ee5d7fd4dcc: Deleting consensus metadata
I20260808 18:22:49.915671 28804 tablet_replica.cc:330] T 55b9a7e9522e43359e6538d69adf909d P 768b4c9b77244208ba40dde9a269778b: stopping tablet replica
I20260808 18:22:49.916043 28804 raft_consensus.cc:2229] T 55b9a7e9522e43359e6538d69adf909d P 768b4c9b77244208ba40dde9a269778b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:22:49.916424 28804 raft_consensus.cc:2258] T 55b9a7e9522e43359e6538d69adf909d P 768b4c9b77244208ba40dde9a269778b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:22:49.916678 28239 catalog_manager.cc:4841] TS 2379ededc56c48c48fef3ee5d7fd4dcc (127.23.72.131:32869): tablet 55b9a7e9522e43359e6538d69adf909d (table test-table [id=9f1e16daa2784bda95fdc5e34ba1918b]) successfully deleted
I20260808 18:22:49.917933 28804 ts_tablet_manager.cc:1905] T 55b9a7e9522e43359e6538d69adf909d P 768b4c9b77244208ba40dde9a269778b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 18:22:49.918349 28238 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.72.129:39301 (error 108)
I20260808 18:22:49.918632 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 28473
I20260808 18:22:49.921983 28769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:49.920612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:39301 (local address 127.0.0.1:45838)
0808 18:22:49.921785 (+ 1173us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:49.921871 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.129:39301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1056}
W20260808 18:22:49.922590 28238 catalog_manager.cc:4584] TS 6c5b8e5968864e07be3476cd7f55d677 (127.23.72.129:39301): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 55b9a7e9522e43359e6538d69adf909d: Network error: Client connection negotiation failed: client connection to 127.23.72.129:39301: connect: Connection refused (error 111)
I20260808 18:22:49.927613 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 28616
I20260808 18:22:49.928020 28769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:49.927682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.130:44627 (local address 127.0.0.1:57782)
0808 18:22:49.927874 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:49.927936 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.130:44627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260808 18:22:49.934846 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 28222
2026-08-08T18:22:49Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2270 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-08T18:22:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:22:49Z Disabled control of system clock
I20260808 18:22:49.967262 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:35585
--webserver_interface=127.23.72.190
--webserver_port=0
--builtin_ntp_servers=127.23.72.148:43777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:35585
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 18:22:50.077965 28811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:50.078235 28811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:50.078272 28811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:50.082362 28811 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:22:50.082422 28811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:50.082437 28811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:50.082453 28811 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:22:50.082469 28811 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:22:50.087175 28811 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:43777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:35585
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:35585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.72.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28811
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:50.088265 28811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:50.089578 28811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:50.097782 28817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:50.097785 28816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:50.097816 28819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:50.098078 28811 server_base.cc:1034] running on GCE node
I20260808 18:22:50.098634 28811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:50.099761 28811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:50.100952 28811 hybrid_clock.cc:648] HybridClock initialized: now 1786213370100932 us; error 32 us; skew 500 ppm
I20260808 18:22:50.108950 28811 webserver.cc:460] Webserver started at http://127.23.72.190:34173/ using document root <none> and password file <none>
I20260808 18:22:50.109548 28811 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:50.109645 28811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:50.109884 28811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:50.112057 28811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "94ae86d0f89a4f828ce2f35dac4b2fd2"
format_stamp: "Formatted at 2026-08-08 18:22:50 on dist-test-slave-5k54"
server_key: "ebe9c90df3b6161a1f59a96f307ee826"
server_key_iv: "a05a53ab725ffcf32791977c385e1162"
server_key_version: "encryptionkey@0"
I20260808 18:22:50.112665 28811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "94ae86d0f89a4f828ce2f35dac4b2fd2"
format_stamp: "Formatted at 2026-08-08 18:22:50 on dist-test-slave-5k54"
server_key: "ebe9c90df3b6161a1f59a96f307ee826"
server_key_iv: "a05a53ab725ffcf32791977c385e1162"
server_key_version: "encryptionkey@0"
I20260808 18:22:50.116319 28811 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:22:50.118844 28825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:50.120157 28811 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:22:50.120302 28811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "94ae86d0f89a4f828ce2f35dac4b2fd2"
format_stamp: "Formatted at 2026-08-08 18:22:50 on dist-test-slave-5k54"
server_key: "ebe9c90df3b6161a1f59a96f307ee826"
server_key_iv: "a05a53ab725ffcf32791977c385e1162"
server_key_version: "encryptionkey@0"
I20260808 18:22:50.120421 28811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:50.125234 28811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:50.125851 28811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:50.126053 28811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:50.133016 28811 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.190:35585
I20260808 18:22:50.133044 28881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.190:35585 every 8 connection(s)
I20260808 18:22:50.133947 28811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:22:50.137167 28882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:50.142395 28882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94ae86d0f89a4f828ce2f35dac4b2fd2: Bootstrap starting.
I20260808 18:22:50.143142 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 28811
I20260808 18:22:50.143291 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:22:50.143641 23842 external_mini_cluster.cc:1428] Setting key c1c3e327d99c3c30357383451a54c20c
I20260808 18:22:50.145115 28882 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 94ae86d0f89a4f828ce2f35dac4b2fd2: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:50.145840 28882 log.cc:824] T 00000000000000000000000000000000 P 94ae86d0f89a4f828ce2f35dac4b2fd2: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:50.148113 28882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94ae86d0f89a4f828ce2f35dac4b2fd2: No bootstrap required, opened a new log
I20260808 18:22:50.148905 28886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:50.144780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59982 (local address 127.23.72.190:35585)
0808 18:22:50.145465 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:50.145482 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:50.145516 (+ 34us) server_negotiation.cc:407] Connection header received
0808 18:22:50.146189 (+ 673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:50.146206 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:50.146478 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:50.146746 (+ 268us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:50.147604 (+ 858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:50.147608 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:50.147620 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:22:50.147630 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:50.147685 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:22:50.147701 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:50.147786 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:50.147946 (+ 160us) server_negotiation.cc:299] Negotiation successful
0808 18:22:50.148226 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":163,"threads_started":1}
I20260808 18:22:50.151722 28882 raft_consensus.cc:348] T 00000000000000000000000000000000 P 94ae86d0f89a4f828ce2f35dac4b2fd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94ae86d0f89a4f828ce2f35dac4b2fd2" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 35585 } }
I20260808 18:22:50.151952 28882 raft_consensus.cc:374] T 00000000000000000000000000000000 P 94ae86d0f89a4f828ce2f35dac4b2fd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:50.152041 28882 raft_consensus.cc:729] T 00000000000000000000000000000000 P 94ae86d0f89a4f828ce2f35dac4b2fd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94ae86d0f89a4f828ce2f35dac4b2fd2, State: Initialized, Role: FOLLOWER
I20260808 18:22:50.152489 28882 consensus_queue.cc:260] T 00000000000000000000000000000000 P 94ae86d0f89a4f828ce2f35dac4b2fd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94ae86d0f89a4f828ce2f35dac4b2fd2" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 35585 } }
I20260808 18:22:50.152647 28882 raft_consensus.cc:388] T 00000000000000000000000000000000 P 94ae86d0f89a4f828ce2f35dac4b2fd2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:22:50.152738 28882 raft_consensus.cc:482] T 00000000000000000000000000000000 P 94ae86d0f89a4f828ce2f35dac4b2fd2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:22:50.152835 28882 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 94ae86d0f89a4f828ce2f35dac4b2fd2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:50.153805 28882 raft_consensus.cc:504] T 00000000000000000000000000000000 P 94ae86d0f89a4f828ce2f35dac4b2fd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94ae86d0f89a4f828ce2f35dac4b2fd2" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 35585 } }
I20260808 18:22:50.154135 28882 leader_election.cc:302] T 00000000000000000000000000000000 P 94ae86d0f89a4f828ce2f35dac4b2fd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 94ae86d0f89a4f828ce2f35dac4b2fd2; no voters:
I20260808 18:22:50.154443 28882 leader_election.cc:288] T 00000000000000000000000000000000 P 94ae86d0f89a4f828ce2f35dac4b2fd2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:22:50.154521 28888 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 94ae86d0f89a4f828ce2f35dac4b2fd2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:22:50.154815 28888 raft_consensus.cc:686] T 00000000000000000000000000000000 P 94ae86d0f89a4f828ce2f35dac4b2fd2 [term 1 LEADER]: Becoming Leader. State: Replica: 94ae86d0f89a4f828ce2f35dac4b2fd2, State: Running, Role: LEADER
I20260808 18:22:50.155112 28888 consensus_queue.cc:237] T 00000000000000000000000000000000 P 94ae86d0f89a4f828ce2f35dac4b2fd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94ae86d0f89a4f828ce2f35dac4b2fd2" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 35585 } }
I20260808 18:22:50.155740 28882 sys_catalog.cc:564] T 00000000000000000000000000000000 P 94ae86d0f89a4f828ce2f35dac4b2fd2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:22:50.157070 28889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94ae86d0f89a4f828ce2f35dac4b2fd2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "94ae86d0f89a4f828ce2f35dac4b2fd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94ae86d0f89a4f828ce2f35dac4b2fd2" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 35585 } } }
I20260808 18:22:50.157189 28889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94ae86d0f89a4f828ce2f35dac4b2fd2 [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:50.157521 28890 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94ae86d0f89a4f828ce2f35dac4b2fd2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94ae86d0f89a4f828ce2f35dac4b2fd2. Latest consensus state: current_term: 1 leader_uuid: "94ae86d0f89a4f828ce2f35dac4b2fd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94ae86d0f89a4f828ce2f35dac4b2fd2" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 35585 } } }
I20260808 18:22:50.157577 28898 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:22:50.157639 28890 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94ae86d0f89a4f828ce2f35dac4b2fd2 [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:50.160600 28898 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:22:50.165637 28898 catalog_manager.cc:1347] Generated new cluster ID: 3a105deac95f4192a30aeaf379c126b9
I20260808 18:22:50.165704 28898 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:22:50.182228 28898 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:22:50.182541 28898 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:22:50.191826 28898 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 94ae86d0f89a4f828ce2f35dac4b2fd2: Generated new TSK 0
I20260808 18:22:50.192517 28898 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:22:50.196348 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.129:0
--local_ip_for_outbound_sockets=127.23.72.129
--webserver_interface=127.23.72.129
--webserver_port=0
--tserver_master_addrs=127.23.72.190:35585
--builtin_ntp_servers=127.23.72.148:43777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 18:22:50.311614 28911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:50.311859 28911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:50.311931 28911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:50.315873 28911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:50.315922 28911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:50.316013 28911 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.129
I20260808 18:22:50.320838 28911 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:43777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.72.129
--webserver_port=0
--tserver_master_addrs=127.23.72.190:35585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.129
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:50.322062 28911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:50.323580 28911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:50.333165 28920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:50.333171 28918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:50.333176 28917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:50.333503 28911 server_base.cc:1034] running on GCE node
I20260808 18:22:50.333861 28911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:50.334442 28911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:50.335595 28911 hybrid_clock.cc:648] HybridClock initialized: now 1786213370335578 us; error 30 us; skew 500 ppm
I20260808 18:22:50.337240 28911 webserver.cc:460] Webserver started at http://127.23.72.129:33821/ using document root <none> and password file <none>
I20260808 18:22:50.337759 28911 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:50.337817 28911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:50.337975 28911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:50.340026 28911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-0/data/instance:
uuid: "86f64719f0b9474cb8e70a87e3199b26"
format_stamp: "Formatted at 2026-08-08 18:22:50 on dist-test-slave-5k54"
server_key: "3bd48fde4d952131d03d99a59411da03"
server_key_iv: "fe27d924f3df82cef2e66b9f2b98e2d0"
server_key_version: "encryptionkey@0"
I20260808 18:22:50.340531 28911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance:
uuid: "86f64719f0b9474cb8e70a87e3199b26"
format_stamp: "Formatted at 2026-08-08 18:22:50 on dist-test-slave-5k54"
server_key: "3bd48fde4d952131d03d99a59411da03"
server_key_iv: "fe27d924f3df82cef2e66b9f2b98e2d0"
server_key_version: "encryptionkey@0"
I20260808 18:22:50.343972 28911 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:22:50.346357 28926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:50.347563 28911 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:22:50.347653 28911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-0/wal
uuid: "86f64719f0b9474cb8e70a87e3199b26"
format_stamp: "Formatted at 2026-08-08 18:22:50 on dist-test-slave-5k54"
server_key: "3bd48fde4d952131d03d99a59411da03"
server_key_iv: "fe27d924f3df82cef2e66b9f2b98e2d0"
server_key_version: "encryptionkey@0"
I20260808 18:22:50.347734 28911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:50.351966 28911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:50.352489 28911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:50.352619 28911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:50.353071 28911 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:22:50.353956 28911 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:22:50.354002 28911 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:50.354038 28911 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:22:50.354053 28911 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:50.363050 28911 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.129:37201
I20260808 18:22:50.363081 29050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.129:37201 every 8 connection(s)
I20260808 18:22:50.363965 28911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
I20260808 18:22:50.369946 28886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:50.365700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:60833 (local address 127.23.72.190:35585)
0808 18:22:50.365846 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:50.365850 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:50.366822 (+ 972us) server_negotiation.cc:407] Connection header received
0808 18:22:50.368290 (+ 1468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:50.368293 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:50.368341 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:50.368458 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:50.369473 (+ 1015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:50.369476 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:50.369478 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:22:50.369490 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:50.369514 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:22:50.369528 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:50.369620 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:50.369765 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 18:22:50.369810 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 18:22:50.370721 29055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:50.366065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:35585 (local address 127.23.72.129:60833)
0808 18:22:50.366679 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:50.366716 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:50.368082 (+ 1366us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:50.368617 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:50.368623 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:50.368706 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:50.369222 (+ 516us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:22:50.369223 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:50.369260 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:22:50.369322 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:50.369615 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:50.369625 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:50.369632 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 18:22:50.369805 (+ 173us) client_negotiation.cc:240] Negotiation successful
0808 18:22:50.370049 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":111,"threads_started":1}
I20260808 18:22:50.371856 29051 heartbeater.cc:347] Connected to a master server at 127.23.72.190:35585
I20260808 18:22:50.372087 29051 heartbeater.cc:464] Registering TS with master...
I20260808 18:22:50.372478 29051 heartbeater.cc:511] Master 127.23.72.190:35585 requested a full tablet report, sending...
I20260808 18:22:50.373755 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 28911
I20260808 18:22:50.373852 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance
I20260808 18:22:50.374017 28842 ts_manager.cc:194] Registered new tserver with Master: 86f64719f0b9474cb8e70a87e3199b26 (127.23.72.129:37201)
I20260808 18:22:50.374186 23842 external_mini_cluster.cc:1428] Setting key 11fea5f467bf0b1bfa17b38fbe3bf029
I20260808 18:22:50.376390 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.130:0
--local_ip_for_outbound_sockets=127.23.72.130
--webserver_interface=127.23.72.130
--webserver_port=0
--tserver_master_addrs=127.23.72.190:35585
--builtin_ntp_servers=127.23.72.148:43777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 18:22:50.488744 29056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:50.489001 29056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:50.489076 29056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:50.493037 29056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:50.493084 29056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:50.493177 29056 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.130
I20260808 18:22:50.497932 29056 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:43777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.72.130
--webserver_port=0
--tserver_master_addrs=127.23.72.190:35585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.130
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:50.498951 29056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:50.500299 29056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:50.509728 29062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:50.509824 29056 server_base.cc:1034] running on GCE node
W20260808 18:22:50.509722 29061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:50.509771 29064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:50.510353 29056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:50.511008 29056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:50.512181 29056 hybrid_clock.cc:648] HybridClock initialized: now 1786213370512169 us; error 24 us; skew 500 ppm
I20260808 18:22:50.513954 29056 webserver.cc:460] Webserver started at http://127.23.72.130:36083/ using document root <none> and password file <none>
I20260808 18:22:50.514523 29056 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:50.514616 29056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:50.514823 29056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:50.516840 29056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-1/data/instance:
uuid: "6c3d0b40676143528b1e27fd3925204e"
format_stamp: "Formatted at 2026-08-08 18:22:50 on dist-test-slave-5k54"
server_key: "53a1cf56d5c4724a1563bf0987c4a340"
server_key_iv: "8435a89e6bec913f68873b454ee0b9d4"
server_key_version: "encryptionkey@0"
I20260808 18:22:50.517391 29056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance:
uuid: "6c3d0b40676143528b1e27fd3925204e"
format_stamp: "Formatted at 2026-08-08 18:22:50 on dist-test-slave-5k54"
server_key: "53a1cf56d5c4724a1563bf0987c4a340"
server_key_iv: "8435a89e6bec913f68873b454ee0b9d4"
server_key_version: "encryptionkey@0"
I20260808 18:22:50.521036 29056 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 18:22:50.523638 29070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:50.524943 29056 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 18:22:50.525086 29056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-1/wal
uuid: "6c3d0b40676143528b1e27fd3925204e"
format_stamp: "Formatted at 2026-08-08 18:22:50 on dist-test-slave-5k54"
server_key: "53a1cf56d5c4724a1563bf0987c4a340"
server_key_iv: "8435a89e6bec913f68873b454ee0b9d4"
server_key_version: "encryptionkey@0"
I20260808 18:22:50.525216 29056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:50.529683 29056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:50.530239 29056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:50.530423 29056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:50.530938 29056 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:22:50.531865 29056 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:22:50.531934 29056 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:50.532016 29056 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:22:50.532063 29056 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:50.540990 29056 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.130:35239
I20260808 18:22:50.541018 29200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.130:35239 every 8 connection(s)
I20260808 18:22:50.541823 29056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
I20260808 18:22:50.542665 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 29056
I20260808 18:22:50.542800 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance
I20260808 18:22:50.543116 23842 external_mini_cluster.cc:1428] Setting key 798be57cffee58603f499523adee896a
I20260808 18:22:50.545423 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.131:0
--local_ip_for_outbound_sockets=127.23.72.131
--webserver_interface=127.23.72.131
--webserver_port=0
--tserver_master_addrs=127.23.72.190:35585
--builtin_ntp_servers=127.23.72.148:43777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260808 18:22:50.549152 28886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:50.544612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:51207 (local address 127.23.72.190:35585)
0808 18:22:50.544786 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:50.544790 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:50.546109 (+ 1319us) server_negotiation.cc:407] Connection header received
0808 18:22:50.547350 (+ 1241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:50.547356 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:50.547402 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:50.547526 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:50.548647 (+ 1121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:50.548654 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:50.548660 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:22:50.548677 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:50.548705 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:22:50.548719 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:50.548818 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:50.548921 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 18:22:50.548967 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 18:22:50.549942 29203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:50.545107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:35585 (local address 127.23.72.130:51207)
0808 18:22:50.545967 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:50.546010 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:50.547154 (+ 1144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:50.547689 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:50.547695 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:50.547779 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:50.548388 (+ 609us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:22:50.548389 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:50.548428 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:22:50.548494 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:50.548809 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:50.548819 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:50.548823 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 18:22:50.548951 (+ 128us) client_negotiation.cc:240] Negotiation successful
0808 18:22:50.549134 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":155,"threads_started":1}
I20260808 18:22:50.551198 29201 heartbeater.cc:347] Connected to a master server at 127.23.72.190:35585
I20260808 18:22:50.551479 29201 heartbeater.cc:464] Registering TS with master...
I20260808 18:22:50.551954 29201 heartbeater.cc:511] Master 127.23.72.190:35585 requested a full tablet report, sending...
I20260808 18:22:50.553012 28841 ts_manager.cc:194] Registered new tserver with Master: 6c3d0b40676143528b1e27fd3925204e (127.23.72.130:35239)
W20260808 18:22:50.657919 29204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:50.658253 29204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:50.658345 29204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:50.662431 29204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:50.662474 29204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:50.662528 29204 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.131
I20260808 18:22:50.667166 29204 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:43777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.72.131
--webserver_port=0
--tserver_master_addrs=127.23.72.190:35585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29204
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.131
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:50.668208 29204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:50.669543 29204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:50.679064 29210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:50.679064 29209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:50.679153 29204 server_base.cc:1034] running on GCE node
W20260808 18:22:50.679159 29213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:50.679792 29204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:50.680409 29204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:50.681653 29204 hybrid_clock.cc:648] HybridClock initialized: now 1786213370681620 us; error 42 us; skew 500 ppm
I20260808 18:22:50.683568 29204 webserver.cc:460] Webserver started at http://127.23.72.131:40097/ using document root <none> and password file <none>
I20260808 18:22:50.684144 29204 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:50.684235 29204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:50.684475 29204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:50.686462 29204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-2/data/instance:
uuid: "4bb7cb13e4d84e7ea3285619b2f8093f"
format_stamp: "Formatted at 2026-08-08 18:22:50 on dist-test-slave-5k54"
server_key: "338a217ffbfa5b563ca0ac7d34da977d"
server_key_iv: "71b0f7841b5a45a6b53a106d5261f056"
server_key_version: "encryptionkey@0"
I20260808 18:22:50.687024 29204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance:
uuid: "4bb7cb13e4d84e7ea3285619b2f8093f"
format_stamp: "Formatted at 2026-08-08 18:22:50 on dist-test-slave-5k54"
server_key: "338a217ffbfa5b563ca0ac7d34da977d"
server_key_iv: "71b0f7841b5a45a6b53a106d5261f056"
server_key_version: "encryptionkey@0"
I20260808 18:22:50.690510 29204 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260808 18:22:50.693130 29221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:50.694331 29204 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:22:50.694445 29204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-2/wal
uuid: "4bb7cb13e4d84e7ea3285619b2f8093f"
format_stamp: "Formatted at 2026-08-08 18:22:50 on dist-test-slave-5k54"
server_key: "338a217ffbfa5b563ca0ac7d34da977d"
server_key_iv: "71b0f7841b5a45a6b53a106d5261f056"
server_key_version: "encryptionkey@0"
I20260808 18:22:50.694546 29204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:50.699052 29204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:50.699651 29204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:50.699824 29204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:50.700320 29204 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:22:50.701184 29204 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:22:50.701228 29204 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:50.701290 29204 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:22:50.701329 29204 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:50.709817 29204 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.131:44925
I20260808 18:22:50.709830 29349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.131:44925 every 8 connection(s)
I20260808 18:22:50.710644 29204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
I20260808 18:22:50.712656 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 29204
I20260808 18:22:50.712785 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance
I20260808 18:22:50.713126 23842 external_mini_cluster.cc:1428] Setting key 19a00b55d1d0717c168a86571ef0bd57
I20260808 18:22:50.717463 28886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:50.712477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:55179 (local address 127.23.72.190:35585)
0808 18:22:50.712621 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:50.712624 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:50.713911 (+ 1287us) server_negotiation.cc:407] Connection header received
0808 18:22:50.715692 (+ 1781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:50.715696 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:50.715740 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:50.715810 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:50.717074 (+ 1264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:50.717077 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:50.717079 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:22:50.717090 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:50.717109 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:22:50.717117 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:50.717186 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:50.717317 (+ 131us) server_negotiation.cc:299] Negotiation successful
0808 18:22:50.717352 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 18:22:50.718336 29352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:50.712905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:35585 (local address 127.23.72.131:55179)
0808 18:22:50.713743 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:50.713797 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:50.715502 (+ 1705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:50.716209 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:50.716216 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:50.716303 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:50.716837 (+ 534us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:22:50.716839 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:50.716876 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:22:50.716939 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:50.717192 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:50.717198 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:50.717202 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 18:22:50.717336 (+ 134us) client_negotiation.cc:240] Negotiation successful
0808 18:22:50.717553 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":135,"threads_started":1}
I20260808 18:22:50.719410 29350 heartbeater.cc:347] Connected to a master server at 127.23.72.190:35585
I20260808 18:22:50.719692 29350 heartbeater.cc:464] Registering TS with master...
I20260808 18:22:50.720052 29350 heartbeater.cc:511] Master 127.23.72.190:35585 requested a full tablet report, sending...
I20260808 18:22:50.720964 28841 ts_manager.cc:194] Registered new tserver with Master: 4bb7cb13e4d84e7ea3285619b2f8093f (127.23.72.131:44925)
I20260808 18:22:50.727187 23842 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:22:50.730948 28886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:50.729226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59986 (local address 127.23.72.190:35585)
0808 18:22:50.729358 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:50.729364 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:50.729559 (+ 195us) server_negotiation.cc:407] Connection header received
0808 18:22:50.729711 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:50.729714 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:50.729755 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:50.729852 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:50.730572 (+ 720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:50.730587 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:50.730590 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:22:50.730601 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:50.730620 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:22:50.730635 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:50.730720 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:50.730789 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 18:22:50.730829 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 18:22:50.733634 28841 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59986:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:22:50.736582 28841 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:22:50.748281 29364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:50.745063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48254 (local address 127.23.72.131:44925)
0808 18:22:50.745996 (+ 933us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:50.746009 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:50.746039 (+ 30us) server_negotiation.cc:407] Connection header received
0808 18:22:50.746108 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:50.746115 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:50.746310 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:50.746509 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:50.747718 (+ 1209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:50.747722 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:50.747731 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:22:50.747739 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:50.747770 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:22:50.747787 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:50.747911 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:50.748077 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 18:22:50.748150 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":815,"thread_start_us":155,"threads_started":1}
I20260808 18:22:50.748256 29363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:50.744707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.131:44925 (local address 127.0.0.1:48254)
0808 18:22:50.745828 (+ 1121us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:50.745884 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:50.746006 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:50.746526 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:50.746532 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:50.746605 (+ 73us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:50.747243 (+ 638us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:22:50.747245 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:50.747269 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:22:50.747339 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:50.747895 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:50.747914 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:50.747920 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 18:22:50.748047 (+ 127us) client_negotiation.cc:240] Negotiation successful
0808 18:22:50.748118 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":946,"thread_start_us":112,"threads_started":1}
I20260808 18:22:50.748811 29366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:50.745380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58134 (local address 127.23.72.130:35239)
0808 18:22:50.746372 (+ 992us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:50.746386 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:50.746416 (+ 30us) server_negotiation.cc:407] Connection header received
0808 18:22:50.746486 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:50.746491 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:50.746675 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:50.747045 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:50.747905 (+ 860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:50.747912 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:50.747926 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:22:50.747939 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:50.747980 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:22:50.747999 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:50.748125 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:50.748599 (+ 474us) server_negotiation.cc:299] Negotiation successful
0808 18:22:50.748667 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":867,"thread_start_us":136,"threads_started":1}
I20260808 18:22:50.749264 29362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:50.745414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:37201 (local address 127.0.0.1:59326)
0808 18:22:50.746180 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:50.746195 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:50.746259 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:50.747485 (+ 1226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:50.747488 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:50.747502 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:50.748637 (+ 1135us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:22:50.748639 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:50.748649 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:22:50.748673 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:50.749033 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:50.749036 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:50.749039 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 18:22:50.749117 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 18:22:50.749152 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":706,"thread_start_us":92,"threads_started":1}
I20260808 18:22:50.749341 29367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:50.745585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59326 (local address 127.23.72.129:37201)
0808 18:22:50.746249 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:50.746259 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:50.746282 (+ 23us) server_negotiation.cc:407] Connection header received
0808 18:22:50.746452 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:50.746460 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:50.746627 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:50.746755 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:50.748821 (+ 2066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:50.748828 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:50.748840 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:22:50.748853 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:50.748910 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:22:50.748953 (+ 43us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:50.749059 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:50.749130 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 18:22:50.749193 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":136,"threads_started":1}
I20260808 18:22:50.749640 29365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:50.744833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.130:35239 (local address 127.0.0.1:58134)
0808 18:22:50.745830 (+ 997us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:50.745884 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:50.746017 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:50.746830 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:50.746835 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:50.746853 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:50.747355 (+ 502us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:22:50.747357 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:50.747366 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:22:50.747390 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:50.748100 (+ 710us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:50.748103 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:50.748106 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 18:22:50.748184 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 18:22:50.748221 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":917,"spinlock_wait_cycles":66304,"thread_start_us":119,"threads_started":1}
I20260808 18:22:50.751405 29277 tablet_service.cc:1467] Processing CreateTablet for tablet 7a4a8cbf70394b9a9cba79364839e739 (DEFAULT_TABLE table=test-table [id=ec89d78b4af345d09763074553d30323]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:22:50.752125 28977 tablet_service.cc:1467] Processing CreateTablet for tablet 7a4a8cbf70394b9a9cba79364839e739 (DEFAULT_TABLE table=test-table [id=ec89d78b4af345d09763074553d30323]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:22:50.752574 29277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a4a8cbf70394b9a9cba79364839e739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:50.752678 29125 tablet_service.cc:1467] Processing CreateTablet for tablet 7a4a8cbf70394b9a9cba79364839e739 (DEFAULT_TABLE table=test-table [id=ec89d78b4af345d09763074553d30323]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:22:50.753146 28977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a4a8cbf70394b9a9cba79364839e739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:50.753566 29125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a4a8cbf70394b9a9cba79364839e739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:50.759706 29368 tablet_bootstrap.cc:492] T 7a4a8cbf70394b9a9cba79364839e739 P 6c3d0b40676143528b1e27fd3925204e: Bootstrap starting.
I20260808 18:22:50.760679 29369 tablet_bootstrap.cc:492] T 7a4a8cbf70394b9a9cba79364839e739 P 86f64719f0b9474cb8e70a87e3199b26: Bootstrap starting.
I20260808 18:22:50.762308 29368 tablet_bootstrap.cc:654] T 7a4a8cbf70394b9a9cba79364839e739 P 6c3d0b40676143528b1e27fd3925204e: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:50.762308 29369 tablet_bootstrap.cc:654] T 7a4a8cbf70394b9a9cba79364839e739 P 86f64719f0b9474cb8e70a87e3199b26: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:50.763047 29369 log.cc:824] T 7a4a8cbf70394b9a9cba79364839e739 P 86f64719f0b9474cb8e70a87e3199b26: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:50.763062 29368 log.cc:824] T 7a4a8cbf70394b9a9cba79364839e739 P 6c3d0b40676143528b1e27fd3925204e: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:50.763780 29370 tablet_bootstrap.cc:492] T 7a4a8cbf70394b9a9cba79364839e739 P 4bb7cb13e4d84e7ea3285619b2f8093f: Bootstrap starting.
I20260808 18:22:50.765129 29369 tablet_bootstrap.cc:492] T 7a4a8cbf70394b9a9cba79364839e739 P 86f64719f0b9474cb8e70a87e3199b26: No bootstrap required, opened a new log
I20260808 18:22:50.765129 29368 tablet_bootstrap.cc:492] T 7a4a8cbf70394b9a9cba79364839e739 P 6c3d0b40676143528b1e27fd3925204e: No bootstrap required, opened a new log
I20260808 18:22:50.765317 29368 ts_tablet_manager.cc:1397] T 7a4a8cbf70394b9a9cba79364839e739 P 6c3d0b40676143528b1e27fd3925204e: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260808 18:22:50.765318 29369 ts_tablet_manager.cc:1397] T 7a4a8cbf70394b9a9cba79364839e739 P 86f64719f0b9474cb8e70a87e3199b26: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 18:22:50.765552 29370 tablet_bootstrap.cc:654] T 7a4a8cbf70394b9a9cba79364839e739 P 4bb7cb13e4d84e7ea3285619b2f8093f: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:50.766372 29370 log.cc:824] T 7a4a8cbf70394b9a9cba79364839e739 P 4bb7cb13e4d84e7ea3285619b2f8093f: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:50.768016 29370 tablet_bootstrap.cc:492] T 7a4a8cbf70394b9a9cba79364839e739 P 4bb7cb13e4d84e7ea3285619b2f8093f: No bootstrap required, opened a new log
I20260808 18:22:50.768205 29370 ts_tablet_manager.cc:1397] T 7a4a8cbf70394b9a9cba79364839e739 P 4bb7cb13e4d84e7ea3285619b2f8093f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 18:22:50.769919 29369 raft_consensus.cc:348] T 7a4a8cbf70394b9a9cba79364839e739 P 86f64719f0b9474cb8e70a87e3199b26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bb7cb13e4d84e7ea3285619b2f8093f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 44925 } } peers { permanent_uuid: "6c3d0b40676143528b1e27fd3925204e" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 35239 } } peers { permanent_uuid: "86f64719f0b9474cb8e70a87e3199b26" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 37201 } }
I20260808 18:22:50.769910 29368 raft_consensus.cc:348] T 7a4a8cbf70394b9a9cba79364839e739 P 6c3d0b40676143528b1e27fd3925204e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bb7cb13e4d84e7ea3285619b2f8093f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 44925 } } peers { permanent_uuid: "6c3d0b40676143528b1e27fd3925204e" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 35239 } } peers { permanent_uuid: "86f64719f0b9474cb8e70a87e3199b26" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 37201 } }
I20260808 18:22:50.770185 29368 raft_consensus.cc:374] T 7a4a8cbf70394b9a9cba79364839e739 P 6c3d0b40676143528b1e27fd3925204e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:50.770185 29369 raft_consensus.cc:374] T 7a4a8cbf70394b9a9cba79364839e739 P 86f64719f0b9474cb8e70a87e3199b26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:50.770242 29368 raft_consensus.cc:729] T 7a4a8cbf70394b9a9cba79364839e739 P 6c3d0b40676143528b1e27fd3925204e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c3d0b40676143528b1e27fd3925204e, State: Initialized, Role: FOLLOWER
I20260808 18:22:50.770242 29369 raft_consensus.cc:729] T 7a4a8cbf70394b9a9cba79364839e739 P 86f64719f0b9474cb8e70a87e3199b26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86f64719f0b9474cb8e70a87e3199b26, State: Initialized, Role: FOLLOWER
I20260808 18:22:50.770721 29368 consensus_queue.cc:260] T 7a4a8cbf70394b9a9cba79364839e739 P 6c3d0b40676143528b1e27fd3925204e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bb7cb13e4d84e7ea3285619b2f8093f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 44925 } } peers { permanent_uuid: "6c3d0b40676143528b1e27fd3925204e" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 35239 } } peers { permanent_uuid: "86f64719f0b9474cb8e70a87e3199b26" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 37201 } }
I20260808 18:22:50.770721 29369 consensus_queue.cc:260] T 7a4a8cbf70394b9a9cba79364839e739 P 86f64719f0b9474cb8e70a87e3199b26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bb7cb13e4d84e7ea3285619b2f8093f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 44925 } } peers { permanent_uuid: "6c3d0b40676143528b1e27fd3925204e" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 35239 } } peers { permanent_uuid: "86f64719f0b9474cb8e70a87e3199b26" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 37201 } }
I20260808 18:22:50.771463 29201 heartbeater.cc:503] Master 127.23.72.190:35585 was elected leader, sending a full tablet report...
I20260808 18:22:50.771695 29051 heartbeater.cc:503] Master 127.23.72.190:35585 was elected leader, sending a full tablet report...
I20260808 18:22:50.771755 29368 ts_tablet_manager.cc:1428] T 7a4a8cbf70394b9a9cba79364839e739 P 6c3d0b40676143528b1e27fd3925204e: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260808 18:22:50.771755 29369 ts_tablet_manager.cc:1428] T 7a4a8cbf70394b9a9cba79364839e739 P 86f64719f0b9474cb8e70a87e3199b26: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260808 18:22:50.773296 29370 raft_consensus.cc:348] T 7a4a8cbf70394b9a9cba79364839e739 P 4bb7cb13e4d84e7ea3285619b2f8093f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bb7cb13e4d84e7ea3285619b2f8093f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 44925 } } peers { permanent_uuid: "6c3d0b40676143528b1e27fd3925204e" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 35239 } } peers { permanent_uuid: "86f64719f0b9474cb8e70a87e3199b26" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 37201 } }
I20260808 18:22:50.773559 29370 raft_consensus.cc:374] T 7a4a8cbf70394b9a9cba79364839e739 P 4bb7cb13e4d84e7ea3285619b2f8093f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:50.773613 29370 raft_consensus.cc:729] T 7a4a8cbf70394b9a9cba79364839e739 P 4bb7cb13e4d84e7ea3285619b2f8093f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bb7cb13e4d84e7ea3285619b2f8093f, State: Initialized, Role: FOLLOWER
I20260808 18:22:50.774021 29370 consensus_queue.cc:260] T 7a4a8cbf70394b9a9cba79364839e739 P 4bb7cb13e4d84e7ea3285619b2f8093f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bb7cb13e4d84e7ea3285619b2f8093f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 44925 } } peers { permanent_uuid: "6c3d0b40676143528b1e27fd3925204e" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 35239 } } peers { permanent_uuid: "86f64719f0b9474cb8e70a87e3199b26" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 37201 } }
I20260808 18:22:50.774995 29350 heartbeater.cc:503] Master 127.23.72.190:35585 was elected leader, sending a full tablet report...
I20260808 18:22:50.776166 29370 ts_tablet_manager.cc:1428] T 7a4a8cbf70394b9a9cba79364839e739 P 4bb7cb13e4d84e7ea3285619b2f8093f: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
W20260808 18:22:50.792698 29202 tablet.cc:2369] T 7a4a8cbf70394b9a9cba79364839e739 P 6c3d0b40676143528b1e27fd3925204e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 18:22:50.865126 29054 tablet.cc:2369] T 7a4a8cbf70394b9a9cba79364839e739 P 86f64719f0b9474cb8e70a87e3199b26: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:22:50.868906 29376 raft_consensus.cc:482] T 7a4a8cbf70394b9a9cba79364839e739 P 4bb7cb13e4d84e7ea3285619b2f8093f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:22:50.869138 29376 raft_consensus.cc:504] T 7a4a8cbf70394b9a9cba79364839e739 P 4bb7cb13e4d84e7ea3285619b2f8093f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bb7cb13e4d84e7ea3285619b2f8093f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 44925 } } peers { permanent_uuid: "6c3d0b40676143528b1e27fd3925204e" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 35239 } } peers { permanent_uuid: "86f64719f0b9474cb8e70a87e3199b26" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 37201 } }
I20260808 18:22:50.870723 29376 leader_election.cc:288] T 7a4a8cbf70394b9a9cba79364839e739 P 4bb7cb13e4d84e7ea3285619b2f8093f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c3d0b40676143528b1e27fd3925204e (127.23.72.130:35239), 86f64719f0b9474cb8e70a87e3199b26 (127.23.72.129:37201)
I20260808 18:22:50.873315 29352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:50.871001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.130:35239 (local address 127.23.72.131:40045)
0808 18:22:50.871148 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:50.871165 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:50.871272 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:50.872081 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:50.872087 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:50.872104 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:50.872679 (+ 575us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:22:50.872681 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:50.872699 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:22:50.872731 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:50.873046 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:50.873049 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:50.873052 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 18:22:50.873141 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 18:22:50.873181 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 18:22:50.873346 29366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:50.871360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:40045 (local address 127.23.72.130:35239)
0808 18:22:50.871748 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:50.871756 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:50.871773 (+ 17us) server_negotiation.cc:407] Connection header received
0808 18:22:50.871826 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:50.871831 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:50.871916 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:50.872008 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:50.872895 (+ 887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:50.872901 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:50.872906 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:22:50.872921 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:50.872947 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:22:50.872961 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:50.873055 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:50.873178 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 18:22:50.873224 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272}
I20260808 18:22:50.874135 29367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:50.871726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:48677 (local address 127.23.72.129:37201)
0808 18:22:50.871896 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:50.871900 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:50.871914 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:22:50.871955 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:50.871958 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:50.871998 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:50.872076 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:50.873254 (+ 1178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:50.873258 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:50.873260 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:22:50.873273 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:50.873316 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:22:50.873331 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:50.873897 (+ 566us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:50.873947 (+ 50us) server_negotiation.cc:299] Negotiation successful
0808 18:22:50.873984 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 18:22:50.874212 29148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a4a8cbf70394b9a9cba79364839e739" candidate_uuid: "4bb7cb13e4d84e7ea3285619b2f8093f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c3d0b40676143528b1e27fd3925204e" is_pre_election: true
I20260808 18:22:50.874538 29148 raft_consensus.cc:2454] T 7a4a8cbf70394b9a9cba79364839e739 P 6c3d0b40676143528b1e27fd3925204e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bb7cb13e4d84e7ea3285619b2f8093f in term 0.
I20260808 18:22:50.875077 29377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:50.871295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:37201 (local address 127.23.72.131:48677)
0808 18:22:50.871652 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:50.871668 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:50.871748 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:50.872143 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:50.872147 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:50.872166 (+ 19us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:50.873000 (+ 834us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:22:50.873002 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:50.873016 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:22:50.873071 (+ 55us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:50.873838 (+ 767us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:50.873840 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:50.873843 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 18:22:50.874834 (+ 991us) client_negotiation.cc:240] Negotiation successful
0808 18:22:50.874880 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":181,"threads_started":1}
I20260808 18:22:50.875730 29223 leader_election.cc:302] T 7a4a8cbf70394b9a9cba79364839e739 P 4bb7cb13e4d84e7ea3285619b2f8093f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bb7cb13e4d84e7ea3285619b2f8093f, 6c3d0b40676143528b1e27fd3925204e; no voters:
I20260808 18:22:50.876010 29376 raft_consensus.cc:2781] T 7a4a8cbf70394b9a9cba79364839e739 P 4bb7cb13e4d84e7ea3285619b2f8093f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:22:50.876112 29376 raft_consensus.cc:482] T 7a4a8cbf70394b9a9cba79364839e739 P 4bb7cb13e4d84e7ea3285619b2f8093f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:22:50.876163 29376 raft_consensus.cc:3037] T 7a4a8cbf70394b9a9cba79364839e739 P 4bb7cb13e4d84e7ea3285619b2f8093f [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:50.876504 29001 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a4a8cbf70394b9a9cba79364839e739" candidate_uuid: "4bb7cb13e4d84e7ea3285619b2f8093f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86f64719f0b9474cb8e70a87e3199b26" is_pre_election: true
I20260808 18:22:50.876835 29001 raft_consensus.cc:2454] T 7a4a8cbf70394b9a9cba79364839e739 P 86f64719f0b9474cb8e70a87e3199b26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bb7cb13e4d84e7ea3285619b2f8093f in term 0.
I20260808 18:22:50.877624 29376 raft_consensus.cc:504] T 7a4a8cbf70394b9a9cba79364839e739 P 4bb7cb13e4d84e7ea3285619b2f8093f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bb7cb13e4d84e7ea3285619b2f8093f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 44925 } } peers { permanent_uuid: "6c3d0b40676143528b1e27fd3925204e" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 35239 } } peers { permanent_uuid: "86f64719f0b9474cb8e70a87e3199b26" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 37201 } }
I20260808 18:22:50.877993 29376 leader_election.cc:288] T 7a4a8cbf70394b9a9cba79364839e739 P 4bb7cb13e4d84e7ea3285619b2f8093f [CANDIDATE]: Term 1 election: Requested vote from peers 6c3d0b40676143528b1e27fd3925204e (127.23.72.130:35239), 86f64719f0b9474cb8e70a87e3199b26 (127.23.72.129:37201)
I20260808 18:22:50.878311 29148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a4a8cbf70394b9a9cba79364839e739" candidate_uuid: "4bb7cb13e4d84e7ea3285619b2f8093f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c3d0b40676143528b1e27fd3925204e"
I20260808 18:22:50.878461 29148 raft_consensus.cc:3037] T 7a4a8cbf70394b9a9cba79364839e739 P 6c3d0b40676143528b1e27fd3925204e [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:50.878522 29001 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a4a8cbf70394b9a9cba79364839e739" candidate_uuid: "4bb7cb13e4d84e7ea3285619b2f8093f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86f64719f0b9474cb8e70a87e3199b26"
I20260808 18:22:50.878645 29001 raft_consensus.cc:3037] T 7a4a8cbf70394b9a9cba79364839e739 P 86f64719f0b9474cb8e70a87e3199b26 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:50.879583 29148 raft_consensus.cc:2454] T 7a4a8cbf70394b9a9cba79364839e739 P 6c3d0b40676143528b1e27fd3925204e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bb7cb13e4d84e7ea3285619b2f8093f in term 1.
I20260808 18:22:50.879781 29001 raft_consensus.cc:2454] T 7a4a8cbf70394b9a9cba79364839e739 P 86f64719f0b9474cb8e70a87e3199b26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bb7cb13e4d84e7ea3285619b2f8093f in term 1.
I20260808 18:22:50.879885 29223 leader_election.cc:302] T 7a4a8cbf70394b9a9cba79364839e739 P 4bb7cb13e4d84e7ea3285619b2f8093f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bb7cb13e4d84e7ea3285619b2f8093f, 6c3d0b40676143528b1e27fd3925204e; no voters:
I20260808 18:22:50.880052 29376 raft_consensus.cc:2781] T 7a4a8cbf70394b9a9cba79364839e739 P 4bb7cb13e4d84e7ea3285619b2f8093f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:22:50.880302 29376 raft_consensus.cc:686] T 7a4a8cbf70394b9a9cba79364839e739 P 4bb7cb13e4d84e7ea3285619b2f8093f [term 1 LEADER]: Becoming Leader. State: Replica: 4bb7cb13e4d84e7ea3285619b2f8093f, State: Running, Role: LEADER
I20260808 18:22:50.880625 29376 consensus_queue.cc:237] T 7a4a8cbf70394b9a9cba79364839e739 P 4bb7cb13e4d84e7ea3285619b2f8093f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bb7cb13e4d84e7ea3285619b2f8093f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 44925 } } peers { permanent_uuid: "6c3d0b40676143528b1e27fd3925204e" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 35239 } } peers { permanent_uuid: "86f64719f0b9474cb8e70a87e3199b26" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 37201 } }
I20260808 18:22:50.883991 28841 catalog_manager.cc:5495] T 7a4a8cbf70394b9a9cba79364839e739 P 4bb7cb13e4d84e7ea3285619b2f8093f reported cstate change: term changed from 0 to 1, leader changed from <none> to 4bb7cb13e4d84e7ea3285619b2f8093f (127.23.72.131). New cstate: current_term: 1 leader_uuid: "4bb7cb13e4d84e7ea3285619b2f8093f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bb7cb13e4d84e7ea3285619b2f8093f" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 44925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c3d0b40676143528b1e27fd3925204e" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 35239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86f64719f0b9474cb8e70a87e3199b26" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 37201 } health_report { overall_health: UNKNOWN } } }
I20260808 18:22:50.898716 29364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:50.897175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48258 (local address 127.23.72.131:44925)
0808 18:22:50.897305 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:50.897312 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:50.897327 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:22:50.897518 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:50.897521 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:50.897588 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:50.897675 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:50.898304 (+ 629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:50.898307 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:50.898310 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:22:50.898321 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:50.898349 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:22:50.898357 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:50.898456 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:50.898537 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 18:22:50.898576 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 18:22:50.904649 29001 raft_consensus.cc:1261] T 7a4a8cbf70394b9a9cba79364839e739 P 86f64719f0b9474cb8e70a87e3199b26 [term 1 FOLLOWER]: Refusing update from remote peer 4bb7cb13e4d84e7ea3285619b2f8093f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:22:50.904650 29148 raft_consensus.cc:1261] T 7a4a8cbf70394b9a9cba79364839e739 P 6c3d0b40676143528b1e27fd3925204e [term 1 FOLLOWER]: Refusing update from remote peer 4bb7cb13e4d84e7ea3285619b2f8093f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:22:50.905412 29376 consensus_queue.cc:1035] T 7a4a8cbf70394b9a9cba79364839e739 P 4bb7cb13e4d84e7ea3285619b2f8093f [LEADER]: Connected to new peer: Peer: permanent_uuid: "86f64719f0b9474cb8e70a87e3199b26" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 37201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:22:50.905611 29378 consensus_queue.cc:1035] T 7a4a8cbf70394b9a9cba79364839e739 P 4bb7cb13e4d84e7ea3285619b2f8093f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c3d0b40676143528b1e27fd3925204e" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 35239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:22:50.915366 29385 mvcc.cc:205] Tried to move back new op lower bound from 7316329967217098752 to 7316329967129460736. Current Snapshot: MvccSnapshot[applied={T|T < 7316329967217098752 or (T in {7316329967217098752})}]
I20260808 18:22:50.915602 29384 mvcc.cc:205] Tried to move back new op lower bound from 7316329967217098752 to 7316329967129460736. Current Snapshot: MvccSnapshot[applied={T|T < 7316329967217098752}]
I20260808 18:22:50.921027 28841 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:22:50.921191 28841 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:22:50.923525 28841 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 94ae86d0f89a4f828ce2f35dac4b2fd2: Sending DeleteTablet for 3 replicas of tablet 7a4a8cbf70394b9a9cba79364839e739
I20260808 18:22:50.924253 29277 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a4a8cbf70394b9a9cba79364839e739 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:22:50 UTC) from {username='slave'} at 127.0.0.1:48254
I20260808 18:22:50.924446 29125 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a4a8cbf70394b9a9cba79364839e739 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:22:50 UTC) from {username='slave'} at 127.0.0.1:58134
I20260808 18:22:50.924731 29395 tablet_replica.cc:330] T 7a4a8cbf70394b9a9cba79364839e739 P 4bb7cb13e4d84e7ea3285619b2f8093f: stopping tablet replica
I20260808 18:22:50.924909 29396 tablet_replica.cc:330] T 7a4a8cbf70394b9a9cba79364839e739 P 6c3d0b40676143528b1e27fd3925204e: stopping tablet replica
I20260808 18:22:50.924945 28977 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a4a8cbf70394b9a9cba79364839e739 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:22:50 UTC) from {username='slave'} at 127.0.0.1:59326
I20260808 18:22:50.925092 29395 raft_consensus.cc:2229] T 7a4a8cbf70394b9a9cba79364839e739 P 4bb7cb13e4d84e7ea3285619b2f8093f [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:22:50.925264 29396 raft_consensus.cc:2229] T 7a4a8cbf70394b9a9cba79364839e739 P 6c3d0b40676143528b1e27fd3925204e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:22:50.925510 29395 raft_consensus.cc:2258] T 7a4a8cbf70394b9a9cba79364839e739 P 4bb7cb13e4d84e7ea3285619b2f8093f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:22:50.925549 29396 raft_consensus.cc:2258] T 7a4a8cbf70394b9a9cba79364839e739 P 6c3d0b40676143528b1e27fd3925204e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:22:50.925908 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 28911
I20260808 18:22:50.926985 29395 ts_tablet_manager.cc:1905] T 7a4a8cbf70394b9a9cba79364839e739 P 4bb7cb13e4d84e7ea3285619b2f8093f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:22:50.927086 29396 ts_tablet_manager.cc:1905] T 7a4a8cbf70394b9a9cba79364839e739 P 6c3d0b40676143528b1e27fd3925204e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:22:50.929836 29396 ts_tablet_manager.cc:1918] T 7a4a8cbf70394b9a9cba79364839e739 P 6c3d0b40676143528b1e27fd3925204e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:22:50.929927 29396 log.cc:1196] T 7a4a8cbf70394b9a9cba79364839e739 P 6c3d0b40676143528b1e27fd3925204e: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-1/wal/wals/7a4a8cbf70394b9a9cba79364839e739
I20260808 18:22:50.930240 29396 ts_tablet_manager.cc:1939] T 7a4a8cbf70394b9a9cba79364839e739 P 6c3d0b40676143528b1e27fd3925204e: Deleting consensus metadata
I20260808 18:22:50.930600 29395 ts_tablet_manager.cc:1918] T 7a4a8cbf70394b9a9cba79364839e739 P 4bb7cb13e4d84e7ea3285619b2f8093f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:22:50.930706 29395 log.cc:1196] T 7a4a8cbf70394b9a9cba79364839e739 P 4bb7cb13e4d84e7ea3285619b2f8093f: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213345431840-23842-0/minicluster-data/ts-2/wal/wals/7a4a8cbf70394b9a9cba79364839e739
I20260808 18:22:50.931056 29395 ts_tablet_manager.cc:1939] T 7a4a8cbf70394b9a9cba79364839e739 P 4bb7cb13e4d84e7ea3285619b2f8093f: Deleting consensus metadata
I20260808 18:22:50.932158 28826 catalog_manager.cc:4841] TS 4bb7cb13e4d84e7ea3285619b2f8093f (127.23.72.131:44925): tablet 7a4a8cbf70394b9a9cba79364839e739 (table test-table [id=ec89d78b4af345d09763074553d30323]) successfully deleted
I20260808 18:22:50.932438 28827 catalog_manager.cc:4841] TS 6c3d0b40676143528b1e27fd3925204e (127.23.72.130:35239): tablet 7a4a8cbf70394b9a9cba79364839e739 (table test-table [id=ec89d78b4af345d09763074553d30323]) successfully deleted
W20260808 18:22:50.938525 28827 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.72.129:37201 (error 108)
I20260808 18:22:50.938714 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 29056
I20260808 18:22:50.940421 29365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:50.940073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:37201 (local address 127.0.0.1:59328)
0808 18:22:50.940244 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:50.940324 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.129:37201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260808 18:22:50.941336 28827 catalog_manager.cc:4584] TS 86f64719f0b9474cb8e70a87e3199b26 (127.23.72.129:37201): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7a4a8cbf70394b9a9cba79364839e739: Network error: Client connection negotiation failed: client connection to 127.23.72.129:37201: connect: Connection refused (error 111)
I20260808 18:22:50.946574 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 29204
I20260808 18:22:50.952214 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 28811
2026-08-08T18:22:50Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1016 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260808 18:22:50.975301 23842 mini_oidc.cc:152] Starting JWKS server
I20260808 18:22:50.976976 23842 webserver.cc:460] Webserver started at https://127.0.0.1:41649/ using document root <none> and password file <none>
I20260808 18:22:50.977144 23842 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 18:22:50.977440 23842 webserver.cc:460] Webserver started at http://127.0.0.1:45703/ using document root <none> and password file <none>
2026-08-08T18:22:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:22:50Z Disabled control of system clock
I20260808 18:22:50.986939 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:37949
--webserver_interface=127.23.72.190
--webserver_port=0
--builtin_ntp_servers=127.23.72.148:46207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:37949
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41649/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213345431840-23842-0/testchainca.cert with env {}
W20260808 18:22:51.096215 29404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:51.096465 29404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:51.096503 29404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:51.100479 29404 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:22:51.100536 29404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:51.100553 29404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:51.100569 29404 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:22:51.100584 29404 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 18:22:51.100600 29404 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 18:22:51.100613 29404 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41649/jwks.json
I20260808 18:22:51.105769 29404 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:46207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:37949
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:37949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41649/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.72.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213345431840-23842-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:51.106711 29404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:51.107656 29404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:51.115820 29414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:51.115850 29411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:51.115833 29412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:51.116253 29404 server_base.cc:1034] running on GCE node
I20260808 18:22:51.116776 29404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:51.117853 29404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:51.119051 29404 hybrid_clock.cc:648] HybridClock initialized: now 1786213371119032 us; error 29 us; skew 500 ppm
I20260808 18:22:51.121009 29404 webserver.cc:460] Webserver started at http://127.23.72.190:34711/ using document root <none> and password file <none>
I20260808 18:22:51.121577 29404 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:51.121670 29404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:51.121892 29404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:51.123932 29404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "7041332361a54e759a01ea6f686065b6"
format_stamp: "Formatted at 2026-08-08 18:22:51 on dist-test-slave-5k54"
server_key: "d2ac5fbdcd031ff14f4193533bfa4bf3"
server_key_iv: "9e8d8c2ffeb0aee871c0d996d75ba214"
server_key_version: "encryptionkey@0"
I20260808 18:22:51.124502 29404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "7041332361a54e759a01ea6f686065b6"
format_stamp: "Formatted at 2026-08-08 18:22:51 on dist-test-slave-5k54"
server_key: "d2ac5fbdcd031ff14f4193533bfa4bf3"
server_key_iv: "9e8d8c2ffeb0aee871c0d996d75ba214"
server_key_version: "encryptionkey@0"
I20260808 18:22:51.128137 29404 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:22:51.130548 29420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:51.131814 29404 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:22:51.131961 29404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "7041332361a54e759a01ea6f686065b6"
format_stamp: "Formatted at 2026-08-08 18:22:51 on dist-test-slave-5k54"
server_key: "d2ac5fbdcd031ff14f4193533bfa4bf3"
server_key_iv: "9e8d8c2ffeb0aee871c0d996d75ba214"
server_key_version: "encryptionkey@0"
I20260808 18:22:51.132071 29404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:51.171619 29404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:51.172363 29404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:51.172583 29404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:51.179953 29404 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.190:37949
I20260808 18:22:51.179960 29477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.190:37949 every 8 connection(s)
I20260808 18:22:51.180913 29404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:22:51.182950 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 29404
I20260808 18:22:51.183084 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:22:51.183410 23842 external_mini_cluster.cc:1428] Setting key f8867597e72935db656bb97911d061d9
I20260808 18:22:51.184582 29478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:51.190047 29478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7041332361a54e759a01ea6f686065b6: Bootstrap starting.
I20260808 18:22:51.192305 29478 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7041332361a54e759a01ea6f686065b6: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:51.192507 29481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:51.184797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41364 (local address 127.23.72.190:37949)
0808 18:22:51.185435 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:51.185451 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:51.185487 (+ 36us) server_negotiation.cc:407] Connection header received
0808 18:22:51.186145 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:51.186162 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:51.186550 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:51.186881 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:51.187364 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:51.188381 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:51.189294 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:51.189765 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:51.189949 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:51.190818 (+ 869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:51.190823 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:51.190835 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:22:51.191203 (+ 368us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:51.191257 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:22:51.191277 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:51.191380 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:51.191531 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 18:22:51.191782 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":149,"threads_started":1}
I20260808 18:22:51.193214 29478 log.cc:824] T 00000000000000000000000000000000 P 7041332361a54e759a01ea6f686065b6: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:51.195159 23842 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:22:51.195374 29478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7041332361a54e759a01ea6f686065b6: No bootstrap required, opened a new log
I20260808 18:22:51.198885 29478 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7041332361a54e759a01ea6f686065b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7041332361a54e759a01ea6f686065b6" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 37949 } }
I20260808 18:22:51.199118 29478 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7041332361a54e759a01ea6f686065b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:51.199204 29478 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7041332361a54e759a01ea6f686065b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7041332361a54e759a01ea6f686065b6, State: Initialized, Role: FOLLOWER
I20260808 18:22:51.199728 29478 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7041332361a54e759a01ea6f686065b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7041332361a54e759a01ea6f686065b6" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 37949 } }
I20260808 18:22:51.199862 29478 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7041332361a54e759a01ea6f686065b6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:22:51.199908 29478 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7041332361a54e759a01ea6f686065b6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:22:51.199977 29478 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7041332361a54e759a01ea6f686065b6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:51.200736 29478 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7041332361a54e759a01ea6f686065b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7041332361a54e759a01ea6f686065b6" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 37949 } }
I20260808 18:22:51.201018 29478 leader_election.cc:302] T 00000000000000000000000000000000 P 7041332361a54e759a01ea6f686065b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7041332361a54e759a01ea6f686065b6; no voters:
I20260808 18:22:51.201251 29478 leader_election.cc:288] T 00000000000000000000000000000000 P 7041332361a54e759a01ea6f686065b6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:22:51.201376 29483 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7041332361a54e759a01ea6f686065b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:22:51.201627 29483 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7041332361a54e759a01ea6f686065b6 [term 1 LEADER]: Becoming Leader. State: Replica: 7041332361a54e759a01ea6f686065b6, State: Running, Role: LEADER
I20260808 18:22:51.201918 29483 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7041332361a54e759a01ea6f686065b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7041332361a54e759a01ea6f686065b6" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 37949 } }
I20260808 18:22:51.202455 29478 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7041332361a54e759a01ea6f686065b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:22:51.203532 29485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7041332361a54e759a01ea6f686065b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7041332361a54e759a01ea6f686065b6. Latest consensus state: current_term: 1 leader_uuid: "7041332361a54e759a01ea6f686065b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7041332361a54e759a01ea6f686065b6" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 37949 } } }
I20260808 18:22:51.203660 29485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7041332361a54e759a01ea6f686065b6 [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:51.203984 29484 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7041332361a54e759a01ea6f686065b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7041332361a54e759a01ea6f686065b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7041332361a54e759a01ea6f686065b6" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 37949 } } }
I20260808 18:22:51.204075 29484 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7041332361a54e759a01ea6f686065b6 [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:51.204193 29493 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:22:51.207018 29493 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:22:51.212493 29493 catalog_manager.cc:1347] Generated new cluster ID: 90d7b5af59da4043a54072eaf26d8f03
I20260808 18:22:51.212554 29493 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:22:51.233132 29493 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:22:51.234340 29493 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:22:51.240717 29493 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7041332361a54e759a01ea6f686065b6: Generated new TSK 0
I20260808 18:22:51.241504 29493 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 18:22:51.275393 29511 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260808 18:22:51.275626 29511 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 18:22:51.276135 29481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:51.256544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41368 (local address 127.23.72.190:37949)
0808 18:22:51.256690 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:51.256695 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:51.256952 (+ 257us) server_negotiation.cc:407] Connection header received
0808 18:22:51.257164 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:51.257166 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:51.257212 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:51.257293 (+ 81us) server_negotiation.cc:226] Negotiated authn=JWT
0808 18:22:51.257565 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:51.258094 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:51.259319 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:51.259618 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:51.259697 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:51.260076 (+ 379us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 18:22:51.275776 (+ 15700us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41649/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 18:22:51.276025 (+ 249us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41368: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41649/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":56}
W20260808 18:22:51.276239 29481 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41368: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41649/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260808 18:22:51.277742 29509 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:22:51.256490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37949 (local address 127.0.0.1:41368)
0808 18:22:51.256857 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:51.256873 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:51.257005 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:51.257331 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:51.257335 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:51.257344 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0808 18:22:51.257463 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:51.257469 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:51.258205 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:51.258208 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:51.259205 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:51.259214 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:51.259731 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:51.259735 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:51.259834 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:51.259896 (+ 62us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 18:22:51.276021 (+ 16125us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41649/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 18:22:51.276274 (+ 253us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.23.72.190:37949: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41649/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":132,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.23.72.190:37949: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41649/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260808 18:22:51.278492 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 29404
2026-08-08T18:22:51Z chronyd exiting
I20260808 18:22:51.317798 23842 test_util.cc:175] -----------------------------------------------
I20260808 18:22:51.317909 23842 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213345431840-23842-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (344 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260808 18:22:51.319043 23842 mini_oidc.cc:152] Starting JWKS server
I20260808 18:22:51.319976 23842 webserver.cc:460] Webserver started at https://127.0.0.1:42671/ using document root <none> and password file <none>
I20260808 18:22:51.320089 23842 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 18:22:51.320387 23842 webserver.cc:460] Webserver started at http://127.0.0.1:42707/ using document root <none> and password file <none>
2026-08-08T18:22:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:22:51Z Disabled control of system clock
I20260808 18:22:51.330567 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:38503
--webserver_interface=127.23.72.190
--webserver_port=0
--builtin_ntp_servers=127.23.72.148:44015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:38503
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42671/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213345431840-23842-0/testchainca.cert with env {}
W20260808 18:22:51.440289 29518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:51.440544 29518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:51.440583 29518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:51.444519 29518 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:22:51.444569 29518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:51.444584 29518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:51.444600 29518 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:22:51.444615 29518 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 18:22:51.444630 29518 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 18:22:51.444643 29518 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42671/jwks.json
I20260808 18:22:51.449829 29518 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:44015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:38503
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:38503
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42671/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.72.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213345431840-23842-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29518
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:51.450752 29518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:51.451683 29518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:51.459939 29524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:51.459936 29523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:51.459939 29526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:51.460186 29518 server_base.cc:1034] running on GCE node
I20260808 18:22:51.460681 29518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:51.461709 29518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:51.462875 29518 hybrid_clock.cc:648] HybridClock initialized: now 1786213371462856 us; error 31 us; skew 500 ppm
I20260808 18:22:51.464696 29518 webserver.cc:460] Webserver started at http://127.23.72.190:44079/ using document root <none> and password file <none>
I20260808 18:22:51.465255 29518 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:51.465314 29518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:51.465518 29518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:51.467595 29518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "bebf31911f9a44c2934effa79ced95c8"
format_stamp: "Formatted at 2026-08-08 18:22:51 on dist-test-slave-5k54"
server_key: "fe8a7bf080162b057965c73fff69d40a"
server_key_iv: "517142d6899a591980cb89e275edbfb8"
server_key_version: "encryptionkey@0"
I20260808 18:22:51.468151 29518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "bebf31911f9a44c2934effa79ced95c8"
format_stamp: "Formatted at 2026-08-08 18:22:51 on dist-test-slave-5k54"
server_key: "fe8a7bf080162b057965c73fff69d40a"
server_key_iv: "517142d6899a591980cb89e275edbfb8"
server_key_version: "encryptionkey@0"
I20260808 18:22:51.471655 29518 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 18:22:51.473938 29533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:51.475033 29518 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:22:51.475154 29518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "bebf31911f9a44c2934effa79ced95c8"
format_stamp: "Formatted at 2026-08-08 18:22:51 on dist-test-slave-5k54"
server_key: "fe8a7bf080162b057965c73fff69d40a"
server_key_iv: "517142d6899a591980cb89e275edbfb8"
server_key_version: "encryptionkey@0"
I20260808 18:22:51.475255 29518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:51.515679 29518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:51.516383 29518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:51.516602 29518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:51.523793 29518 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.190:38503
I20260808 18:22:51.523808 29591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.190:38503 every 8 connection(s)
I20260808 18:22:51.524899 29518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:22:51.526765 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 29518
I20260808 18:22:51.526870 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:22:51.527158 23842 external_mini_cluster.cc:1428] Setting key d4a051daaa3c012f534fed15d543fe20
I20260808 18:22:51.528486 29592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:51.534806 29592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bebf31911f9a44c2934effa79ced95c8: Bootstrap starting.
I20260808 18:22:51.536379 29595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:51.528538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60042 (local address 127.23.72.190:38503)
0808 18:22:51.529201 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:51.529217 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:51.529252 (+ 35us) server_negotiation.cc:407] Connection header received
0808 18:22:51.529901 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:51.529916 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:51.530285 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:51.530566 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:51.530921 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:51.531927 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:51.533130 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:51.533582 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:51.533692 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:51.534597 (+ 905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:51.534602 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:51.534613 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:22:51.535048 (+ 435us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:51.535102 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:22:51.535122 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:51.535195 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:51.535367 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 18:22:51.535677 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":159,"threads_started":1}
I20260808 18:22:51.537209 29592 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bebf31911f9a44c2934effa79ced95c8: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:51.538080 29592 log.cc:824] T 00000000000000000000000000000000 P bebf31911f9a44c2934effa79ced95c8: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:51.539211 23842 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:22:51.540362 29592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bebf31911f9a44c2934effa79ced95c8: No bootstrap required, opened a new log
I20260808 18:22:51.544219 29592 raft_consensus.cc:348] T 00000000000000000000000000000000 P bebf31911f9a44c2934effa79ced95c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bebf31911f9a44c2934effa79ced95c8" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 38503 } }
I20260808 18:22:51.544450 29592 raft_consensus.cc:374] T 00000000000000000000000000000000 P bebf31911f9a44c2934effa79ced95c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:51.544486 29592 raft_consensus.cc:729] T 00000000000000000000000000000000 P bebf31911f9a44c2934effa79ced95c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bebf31911f9a44c2934effa79ced95c8, State: Initialized, Role: FOLLOWER
I20260808 18:22:51.544844 29592 consensus_queue.cc:260] T 00000000000000000000000000000000 P bebf31911f9a44c2934effa79ced95c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bebf31911f9a44c2934effa79ced95c8" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 38503 } }
I20260808 18:22:51.544961 29592 raft_consensus.cc:388] T 00000000000000000000000000000000 P bebf31911f9a44c2934effa79ced95c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:22:51.545007 29592 raft_consensus.cc:482] T 00000000000000000000000000000000 P bebf31911f9a44c2934effa79ced95c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:22:51.545075 29592 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bebf31911f9a44c2934effa79ced95c8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:51.545879 29592 raft_consensus.cc:504] T 00000000000000000000000000000000 P bebf31911f9a44c2934effa79ced95c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bebf31911f9a44c2934effa79ced95c8" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 38503 } }
I20260808 18:22:51.546167 29592 leader_election.cc:302] T 00000000000000000000000000000000 P bebf31911f9a44c2934effa79ced95c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bebf31911f9a44c2934effa79ced95c8; no voters:
I20260808 18:22:51.546408 29592 leader_election.cc:288] T 00000000000000000000000000000000 P bebf31911f9a44c2934effa79ced95c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:22:51.546532 29597 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bebf31911f9a44c2934effa79ced95c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:22:51.546803 29597 raft_consensus.cc:686] T 00000000000000000000000000000000 P bebf31911f9a44c2934effa79ced95c8 [term 1 LEADER]: Becoming Leader. State: Replica: bebf31911f9a44c2934effa79ced95c8, State: Running, Role: LEADER
I20260808 18:22:51.547108 29597 consensus_queue.cc:237] T 00000000000000000000000000000000 P bebf31911f9a44c2934effa79ced95c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bebf31911f9a44c2934effa79ced95c8" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 38503 } }
I20260808 18:22:51.547653 29592 sys_catalog.cc:564] T 00000000000000000000000000000000 P bebf31911f9a44c2934effa79ced95c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:22:51.548831 29599 sys_catalog.cc:455] T 00000000000000000000000000000000 P bebf31911f9a44c2934effa79ced95c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bebf31911f9a44c2934effa79ced95c8. Latest consensus state: current_term: 1 leader_uuid: "bebf31911f9a44c2934effa79ced95c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bebf31911f9a44c2934effa79ced95c8" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 38503 } } }
I20260808 18:22:51.548847 29598 sys_catalog.cc:455] T 00000000000000000000000000000000 P bebf31911f9a44c2934effa79ced95c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bebf31911f9a44c2934effa79ced95c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bebf31911f9a44c2934effa79ced95c8" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 38503 } } }
I20260808 18:22:51.549000 29599 sys_catalog.cc:458] T 00000000000000000000000000000000 P bebf31911f9a44c2934effa79ced95c8 [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:51.549017 29598 sys_catalog.cc:458] T 00000000000000000000000000000000 P bebf31911f9a44c2934effa79ced95c8 [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:51.549871 29609 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:22:51.552629 29609 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:22:51.557893 29609 catalog_manager.cc:1347] Generated new cluster ID: 0b051de6b8764180bdd26661936a88ad
I20260808 18:22:51.557955 29609 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:22:51.580899 29609 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:22:51.581768 29609 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:22:51.587347 29609 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bebf31911f9a44c2934effa79ced95c8: Generated new TSK 0
I20260808 18:22:51.588021 29609 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 18:22:51.612820 29626 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260808 18:22:51.612922 29626 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 18:22:51.613619 29595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:51.597914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60052 (local address 127.23.72.190:38503)
0808 18:22:51.598050 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:51.598055 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:51.598246 (+ 191us) server_negotiation.cc:407] Connection header received
0808 18:22:51.598362 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:51.598365 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:51.598420 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:51.598534 (+ 114us) server_negotiation.cc:226] Negotiated authn=JWT
0808 18:22:51.598749 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:51.599592 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:51.600424 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:51.600766 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:51.600868 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:51.601145 (+ 277us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 18:22:51.613211 (+ 12066us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42671/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 18:22:51.613474 (+ 263us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60052: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42671/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":59}
W20260808 18:22:51.613685 29595 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60052: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42671/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260808 18:22:51.613718 29625 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:22:51.597853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:38503 (local address 127.0.0.1:60052)
0808 18:22:51.598154 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:51.598167 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:51.598257 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:51.598534 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:51.598536 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:51.598543 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0808 18:22:51.598635 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:51.598640 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:51.599693 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:51.599695 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:51.600301 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:51.600307 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:51.600886 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:51.600888 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:51.600949 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:51.600970 (+ 21us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 18:22:51.613399 (+ 12429us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42671/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 18:22:51.613552 (+ 153us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.23.72.190:38503: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42671/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":122,"threads_started":1}
I20260808 18:22:51.614461 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 29518
2026-08-08T18:22:51Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (336 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260808 18:22:51.655189 23842 mini_oidc.cc:152] Starting JWKS server
I20260808 18:22:51.656131 23842 webserver.cc:460] Webserver started at https://127.0.0.1:36925/ using document root <none> and password file <none>
I20260808 18:22:51.656219 23842 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 18:22:51.656441 23842 webserver.cc:460] Webserver started at http://127.0.0.1:45757/ using document root <none> and password file <none>
2026-08-08T18:22:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:22:51Z Disabled control of system clock
I20260808 18:22:51.666427 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:38225
--webserver_interface=127.23.72.190
--webserver_port=0
--builtin_ntp_servers=127.23.72.148:42103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:38225
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36925/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 18:22:51.777673 29633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:51.777925 29633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:51.777963 29633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:51.782054 29633 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:22:51.782107 29633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:51.782122 29633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:51.782140 29633 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:22:51.782155 29633 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 18:22:51.782207 29633 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 18:22:51.782234 29633 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36925/jwks.json
I20260808 18:22:51.786942 29633 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:42103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:38225
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:38225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36925/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.72.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29633
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:51.788017 29633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:51.789399 29633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:51.798214 29641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:51.798225 29638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:51.798472 29633 server_base.cc:1034] running on GCE node
W20260808 18:22:51.798224 29639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:51.799154 29633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:51.800278 29633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:51.801448 29633 hybrid_clock.cc:648] HybridClock initialized: now 1786213371801419 us; error 41 us; skew 500 ppm
I20260808 18:22:51.803237 29633 webserver.cc:460] Webserver started at http://127.23.72.190:33145/ using document root <none> and password file <none>
I20260808 18:22:51.803853 29633 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:51.803951 29633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:51.804162 29633 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:51.806162 29633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "a0b18513aa9a4640bc5b44675526712b"
format_stamp: "Formatted at 2026-08-08 18:22:51 on dist-test-slave-5k54"
server_key: "6471370d83944aaca69881f40541ed08"
server_key_iv: "4543139d8a5d0238f2d6d779e363384f"
server_key_version: "encryptionkey@0"
I20260808 18:22:51.806734 29633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "a0b18513aa9a4640bc5b44675526712b"
format_stamp: "Formatted at 2026-08-08 18:22:51 on dist-test-slave-5k54"
server_key: "6471370d83944aaca69881f40541ed08"
server_key_iv: "4543139d8a5d0238f2d6d779e363384f"
server_key_version: "encryptionkey@0"
I20260808 18:22:51.810333 29633 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 18:22:51.812816 29647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:51.814043 29633 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:22:51.814181 29633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "a0b18513aa9a4640bc5b44675526712b"
format_stamp: "Formatted at 2026-08-08 18:22:51 on dist-test-slave-5k54"
server_key: "6471370d83944aaca69881f40541ed08"
server_key_iv: "4543139d8a5d0238f2d6d779e363384f"
server_key_version: "encryptionkey@0"
I20260808 18:22:51.814294 29633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:51.843321 29633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:51.844357 29633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:51.844570 29633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:51.852242 29633 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.190:38225
I20260808 18:22:51.852254 29705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.190:38225 every 8 connection(s)
I20260808 18:22:51.853219 29633 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:22:51.856385 29707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:51.861913 29707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0b18513aa9a4640bc5b44675526712b: Bootstrap starting.
I20260808 18:22:51.862572 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 29633
I20260808 18:22:51.862709 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:22:51.863009 23842 external_mini_cluster.cc:1428] Setting key 4e5b1d27a9be60868cb2abde2f6bc722
I20260808 18:22:51.864957 29707 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0b18513aa9a4640bc5b44675526712b: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:51.865707 29707 log.cc:824] T 00000000000000000000000000000000 P a0b18513aa9a4640bc5b44675526712b: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:51.867805 29707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0b18513aa9a4640bc5b44675526712b: No bootstrap required, opened a new log
I20260808 18:22:51.871361 29707 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0b18513aa9a4640bc5b44675526712b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0b18513aa9a4640bc5b44675526712b" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 38225 } }
I20260808 18:22:51.871680 29707 raft_consensus.cc:374] T 00000000000000000000000000000000 P a0b18513aa9a4640bc5b44675526712b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:51.871742 29707 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0b18513aa9a4640bc5b44675526712b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0b18513aa9a4640bc5b44675526712b, State: Initialized, Role: FOLLOWER
I20260808 18:22:51.872232 29707 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0b18513aa9a4640bc5b44675526712b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0b18513aa9a4640bc5b44675526712b" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 38225 } }
I20260808 18:22:51.872406 29707 raft_consensus.cc:388] T 00000000000000000000000000000000 P a0b18513aa9a4640bc5b44675526712b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:22:51.872496 29707 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0b18513aa9a4640bc5b44675526712b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:22:51.872597 29707 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0b18513aa9a4640bc5b44675526712b [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:51.872887 29710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:51.864390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55574 (local address 127.23.72.190:38225)
0808 18:22:51.865187 (+ 797us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:51.865204 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:51.865244 (+ 40us) server_negotiation.cc:407] Connection header received
0808 18:22:51.865941 (+ 697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:51.865959 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:51.866365 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:51.866711 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:51.867168 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:51.868256 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:51.869314 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:51.869811 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:51.869989 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:51.870907 (+ 918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:51.870911 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:51.870923 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:22:51.871314 (+ 391us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:51.871370 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:22:51.871390 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:51.871552 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:51.871671 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 18:22:51.871984 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":143,"threads_started":1}
I20260808 18:22:51.873759 29707 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0b18513aa9a4640bc5b44675526712b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0b18513aa9a4640bc5b44675526712b" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 38225 } }
I20260808 18:22:51.874157 29707 leader_election.cc:302] T 00000000000000000000000000000000 P a0b18513aa9a4640bc5b44675526712b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0b18513aa9a4640bc5b44675526712b; no voters:
I20260808 18:22:51.874505 29707 leader_election.cc:288] T 00000000000000000000000000000000 P a0b18513aa9a4640bc5b44675526712b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:22:51.874605 29712 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0b18513aa9a4640bc5b44675526712b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:22:51.874861 29712 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0b18513aa9a4640bc5b44675526712b [term 1 LEADER]: Becoming Leader. State: Replica: a0b18513aa9a4640bc5b44675526712b, State: Running, Role: LEADER
I20260808 18:22:51.875165 29712 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0b18513aa9a4640bc5b44675526712b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0b18513aa9a4640bc5b44675526712b" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 38225 } }
I20260808 18:22:51.875916 29707 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0b18513aa9a4640bc5b44675526712b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:22:51.876282 23842 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:22:51.876394 29712 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0b18513aa9a4640bc5b44675526712b [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0b18513aa9a4640bc5b44675526712b. Latest consensus state: current_term: 1 leader_uuid: "a0b18513aa9a4640bc5b44675526712b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0b18513aa9a4640bc5b44675526712b" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 38225 } } }
I20260808 18:22:51.876503 29712 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0b18513aa9a4640bc5b44675526712b [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:51.878432 29717 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:22:51.878587 29713 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0b18513aa9a4640bc5b44675526712b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a0b18513aa9a4640bc5b44675526712b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0b18513aa9a4640bc5b44675526712b" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 38225 } } }
I20260808 18:22:51.878683 29713 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0b18513aa9a4640bc5b44675526712b [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:51.881253 29717 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:22:51.886307 29717 catalog_manager.cc:1347] Generated new cluster ID: f6deb0d475374a7888501d1d86824808
I20260808 18:22:51.886377 29717 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:22:51.895190 29717 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:22:51.896150 29717 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:22:51.900306 29717 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a0b18513aa9a4640bc5b44675526712b: Generated new TSK 0
I20260808 18:22:51.901109 29717 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:22:51.921752 29710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:51.916480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55586 (local address 127.23.72.190:38225)
0808 18:22:51.916619 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:51.916626 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:51.916819 (+ 193us) server_negotiation.cc:407] Connection header received
0808 18:22:51.916985 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:51.916987 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:51.917032 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:51.917111 (+ 79us) server_negotiation.cc:226] Negotiated authn=JWT
0808 18:22:51.917416 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:51.917948 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:51.919249 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:51.919552 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:51.919643 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:51.919981 (+ 338us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 18:22:51.921404 (+ 1423us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36925/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 18:22:51.921628 (+ 224us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55586: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36925/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":58}
W20260808 18:22:51.921828 29710 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55586: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36925/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260808 18:22:51.923658 29739 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:22:51.916406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:38225 (local address 127.0.0.1:55586)
0808 18:22:51.916712 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:51.916727 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:51.916854 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:51.917139 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:51.917142 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:51.917151 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0808 18:22:51.917262 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:51.917269 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:51.918058 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:51.918062 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:51.919108 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:51.919118 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:51.919680 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:51.919684 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:51.919786 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:51.919815 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 18:22:51.921551 (+ 1736us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36925/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 18:22:51.923473 (+ 1922us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.23.72.190:38225: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36925/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":218,"mutex_wait_us":1711,"thread_start_us":103,"threads_started":1}
I20260808 18:22:51.924589 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 29633
E20260808 18:22:51.938329 29740 webserver.cc:533] Webserver: sslize failed:
E20260808 18:22:51.938406 29740 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-08T18:22:51Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (313 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-08T18:22:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:22:51Z Disabled control of system clock
I20260808 18:22:51.978276 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:39639
--webserver_interface=127.23.72.190
--webserver_port=0
--builtin_ntp_servers=127.23.72.148:34169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:39639
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213345431840-23842-0/testchainca.cert with env {}
W20260808 18:22:52.089234 29744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:52.089489 29744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:52.089529 29744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:52.093494 29744 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:22:52.093541 29744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:52.093556 29744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:52.093573 29744 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:22:52.093588 29744 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 18:22:52.093604 29744 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 18:22:52.093617 29744 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260808 18:22:52.098755 29744 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:34169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:39639
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:39639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.72.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213345431840-23842-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29744
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:52.099867 29744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:52.100898 29744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:52.109745 29750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:52.109812 29744 server_base.cc:1034] running on GCE node
W20260808 18:22:52.109745 29752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:52.109788 29749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:52.110548 29744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:52.111769 29744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:52.112967 29744 hybrid_clock.cc:648] HybridClock initialized: now 1786213372112936 us; error 43 us; skew 500 ppm
I20260808 18:22:52.114918 29744 webserver.cc:460] Webserver started at http://127.23.72.190:35401/ using document root <none> and password file <none>
I20260808 18:22:52.115573 29744 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:52.115669 29744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:52.115909 29744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:52.118002 29744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "0d1df002958f4a4c8f2ba190d4e8cbf9"
format_stamp: "Formatted at 2026-08-08 18:22:52 on dist-test-slave-5k54"
server_key: "9dc874cf534223ffb6899a28b98e9124"
server_key_iv: "9ec294306c6984d39d695ccf64f31ceb"
server_key_version: "encryptionkey@0"
I20260808 18:22:52.118602 29744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "0d1df002958f4a4c8f2ba190d4e8cbf9"
format_stamp: "Formatted at 2026-08-08 18:22:52 on dist-test-slave-5k54"
server_key: "9dc874cf534223ffb6899a28b98e9124"
server_key_iv: "9ec294306c6984d39d695ccf64f31ceb"
server_key_version: "encryptionkey@0"
I20260808 18:22:52.122334 29744 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 18:22:52.124943 29760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:52.126196 29744 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:22:52.126358 29744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "0d1df002958f4a4c8f2ba190d4e8cbf9"
format_stamp: "Formatted at 2026-08-08 18:22:52 on dist-test-slave-5k54"
server_key: "9dc874cf534223ffb6899a28b98e9124"
server_key_iv: "9ec294306c6984d39d695ccf64f31ceb"
server_key_version: "encryptionkey@0"
I20260808 18:22:52.126500 29744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:52.138664 29744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:52.139364 29744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:52.139604 29744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:52.146669 29744 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.190:39639
I20260808 18:22:52.146682 29819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.190:39639 every 8 connection(s)
I20260808 18:22:52.147697 29744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:22:52.150545 29820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:52.154238 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 29744
I20260808 18:22:52.154392 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:22:52.154726 23842 external_mini_cluster.cc:1428] Setting key b7e25ee5796809d59ca3b00293a4bb0e
I20260808 18:22:52.156615 29820 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d1df002958f4a4c8f2ba190d4e8cbf9: Bootstrap starting.
I20260808 18:22:52.158963 29820 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d1df002958f4a4c8f2ba190d4e8cbf9: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:52.159786 29820 log.cc:824] T 00000000000000000000000000000000 P 0d1df002958f4a4c8f2ba190d4e8cbf9: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:52.161842 29820 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d1df002958f4a4c8f2ba190d4e8cbf9: No bootstrap required, opened a new log
I20260808 18:22:52.164681 29823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:52.156324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48912 (local address 127.23.72.190:39639)
0808 18:22:52.156980 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:52.156997 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:52.157035 (+ 38us) server_negotiation.cc:407] Connection header received
0808 18:22:52.157718 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:52.157736 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:52.158142 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:52.158449 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:52.158814 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:52.160046 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:52.161092 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:52.161566 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:52.161751 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:52.162635 (+ 884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:52.162640 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:52.162658 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:22:52.163044 (+ 386us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:52.163100 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:22:52.163120 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:52.163232 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:52.163388 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 18:22:52.163663 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":144,"threads_started":1}
I20260808 18:22:52.165362 29820 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d1df002958f4a4c8f2ba190d4e8cbf9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d1df002958f4a4c8f2ba190d4e8cbf9" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 39639 } }
I20260808 18:22:52.165640 29820 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0d1df002958f4a4c8f2ba190d4e8cbf9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:52.165699 29820 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d1df002958f4a4c8f2ba190d4e8cbf9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d1df002958f4a4c8f2ba190d4e8cbf9, State: Initialized, Role: FOLLOWER
I20260808 18:22:52.166175 29820 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d1df002958f4a4c8f2ba190d4e8cbf9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d1df002958f4a4c8f2ba190d4e8cbf9" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 39639 } }
I20260808 18:22:52.166308 29820 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d1df002958f4a4c8f2ba190d4e8cbf9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:22:52.166353 29820 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d1df002958f4a4c8f2ba190d4e8cbf9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:22:52.166458 29820 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d1df002958f4a4c8f2ba190d4e8cbf9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:52.167678 29820 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d1df002958f4a4c8f2ba190d4e8cbf9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d1df002958f4a4c8f2ba190d4e8cbf9" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 39639 } }
I20260808 18:22:52.167747 23842 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:22:52.168059 29820 leader_election.cc:302] T 00000000000000000000000000000000 P 0d1df002958f4a4c8f2ba190d4e8cbf9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d1df002958f4a4c8f2ba190d4e8cbf9; no voters:
I20260808 18:22:52.168433 29820 leader_election.cc:288] T 00000000000000000000000000000000 P 0d1df002958f4a4c8f2ba190d4e8cbf9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:22:52.168540 29825 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d1df002958f4a4c8f2ba190d4e8cbf9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:22:52.168766 29825 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d1df002958f4a4c8f2ba190d4e8cbf9 [term 1 LEADER]: Becoming Leader. State: Replica: 0d1df002958f4a4c8f2ba190d4e8cbf9, State: Running, Role: LEADER
I20260808 18:22:52.169068 29825 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d1df002958f4a4c8f2ba190d4e8cbf9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d1df002958f4a4c8f2ba190d4e8cbf9" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 39639 } }
I20260808 18:22:52.169854 29820 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d1df002958f4a4c8f2ba190d4e8cbf9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:22:52.170926 29827 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d1df002958f4a4c8f2ba190d4e8cbf9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0d1df002958f4a4c8f2ba190d4e8cbf9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d1df002958f4a4c8f2ba190d4e8cbf9" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 39639 } } }
I20260808 18:22:52.171099 29827 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d1df002958f4a4c8f2ba190d4e8cbf9 [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:52.171525 29835 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:22:52.171376 29826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d1df002958f4a4c8f2ba190d4e8cbf9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d1df002958f4a4c8f2ba190d4e8cbf9. Latest consensus state: current_term: 1 leader_uuid: "0d1df002958f4a4c8f2ba190d4e8cbf9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d1df002958f4a4c8f2ba190d4e8cbf9" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 39639 } } }
I20260808 18:22:52.172216 29826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d1df002958f4a4c8f2ba190d4e8cbf9 [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:52.174217 29835 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:22:52.179527 29835 catalog_manager.cc:1347] Generated new cluster ID: bdb54d1cb9ad4de58015e6b18b003b77
I20260808 18:22:52.179597 29835 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:22:52.201313 29835 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:22:52.202227 29835 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:22:52.211386 29835 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d1df002958f4a4c8f2ba190d4e8cbf9: Generated new TSK 0
I20260808 18:22:52.212107 29835 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:22:52.276022 29823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:52.220915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48916 (local address 127.23.72.190:39639)
0808 18:22:52.221038 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:52.221043 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:52.221207 (+ 164us) server_negotiation.cc:407] Connection header received
0808 18:22:52.221358 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:52.221360 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:52.221422 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:52.221509 (+ 87us) server_negotiation.cc:226] Negotiated authn=JWT
0808 18:22:52.221769 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:52.222288 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:52.223566 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:52.223788 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:52.223867 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:52.224188 (+ 321us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 18:22:52.275642 (+ 51454us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 18:22:52.275902 (+ 260us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48916: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":38}
W20260808 18:22:52.276122 29823 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48916: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260808 18:22:52.276588 29852 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:22:52.220807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:39639 (local address 127.0.0.1:48916)
0808 18:22:52.221116 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:52.221132 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:52.221244 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:52.221540 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:52.221543 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:52.221551 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0808 18:22:52.221667 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:52.221673 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:52.222405 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:52.222408 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:52.223454 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:52.223464 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:52.223916 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:52.223920 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:52.224004 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:52.224029 (+ 25us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 18:22:52.275884 (+ 51855us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 18:22:52.276110 (+ 226us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.23.72.190:39639: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":112,"threads_started":1}
I20260808 18:22:52.277707 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 29744
2026-08-08T18:22:52Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (330 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (122 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (123 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:22:52 dist-test-slave-5k54 krb5kdc[29864](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:22:52 dist-test-slave-5k54 krb5kdc[29864](info): set up 2 sockets
Aug 08 18:22:52 dist-test-slave-5k54 krb5kdc[29864](info): commencing operation
krb5kdc: starting...
W20260808 18:22:53.601787 23842 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Aug 08 18:22:53 dist-test-slave-5k54 krb5kdc[29864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:22:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:22:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.72.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
I20260808 18:22:53.735595 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:45591
--webserver_interface=127.23.72.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:44187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:45591
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:53.871838 29878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:53.872115 29878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:53.872156 29878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:53.876191 29878 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:22:53.876246 29878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:53.876266 29878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:53.876281 29878 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:22:53.876340 29878 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:22:53.881596 29878 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:44187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:45591
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:45591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.72.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29878
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:53.882692 29878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:53.883769 29878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:53.892279 29886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:53.892302 29883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:53.892374 29878 server_base.cc:1034] running on GCE node
W20260808 18:22:53.892302 29884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:53.893108 29878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:53.894178 29878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:53.895355 29878 hybrid_clock.cc:648] HybridClock initialized: now 1786213373895341 us; error 26 us; skew 500 ppm
Aug 08 18:22:53 dist-test-slave-5k54 krb5kdc[29864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213373, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:53.898398 29878 init.cc:377] Logged in from keytab as kudu/127.23.72.190@KRBTEST.COM (short username kudu)
I20260808 18:22:53.899271 29878 webserver.cc:460] Webserver started at http://127.23.72.190:38207/ using document root <none> and password file <none>
I20260808 18:22:53.899883 29878 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:53.899951 29878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:53.900121 29878 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:53.902288 29878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "263eef1635e148f589136260fbf424f5"
format_stamp: "Formatted at 2026-08-08 18:22:53 on dist-test-slave-5k54"
server_key: "2352b3d169723e2cdb7aa5ee82d72f6f"
server_key_iv: "7ed465ef40ed9ed5d1ff4e53470aaa2d"
server_key_version: "encryptionkey@0"
I20260808 18:22:53.902859 29878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "263eef1635e148f589136260fbf424f5"
format_stamp: "Formatted at 2026-08-08 18:22:53 on dist-test-slave-5k54"
server_key: "2352b3d169723e2cdb7aa5ee82d72f6f"
server_key_iv: "7ed465ef40ed9ed5d1ff4e53470aaa2d"
server_key_version: "encryptionkey@0"
I20260808 18:22:53.906821 29878 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260808 18:22:53.909381 29894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:53.910723 29878 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:22:53.910903 29878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "263eef1635e148f589136260fbf424f5"
format_stamp: "Formatted at 2026-08-08 18:22:53 on dist-test-slave-5k54"
server_key: "2352b3d169723e2cdb7aa5ee82d72f6f"
server_key_iv: "7ed465ef40ed9ed5d1ff4e53470aaa2d"
server_key_version: "encryptionkey@0"
I20260808 18:22:53.911026 29878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:53.925318 29878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:53.928580 29878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:53.928831 29878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:53.936408 29878 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.190:45591
I20260808 18:22:53.936517 29949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.190:45591 every 8 connection(s)
I20260808 18:22:53.937419 29878 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:22:53.940368 29950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:53.942831 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 29878
I20260808 18:22:53.943002 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:22:53.943320 23842 external_mini_cluster.cc:1428] Setting key 097899fb43581406f1508fc4a8fd0545
I20260808 18:22:53.946213 29950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 263eef1635e148f589136260fbf424f5: Bootstrap starting.
I20260808 18:22:53.948974 29950 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 263eef1635e148f589136260fbf424f5: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:53.949888 29950 log.cc:824] T 00000000000000000000000000000000 P 263eef1635e148f589136260fbf424f5: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:53.951977 29950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 263eef1635e148f589136260fbf424f5: No bootstrap required, opened a new log
Aug 08 18:22:53 dist-test-slave-5k54 krb5kdc[29864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:22:53.955219 29950 raft_consensus.cc:348] T 00000000000000000000000000000000 P 263eef1635e148f589136260fbf424f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "263eef1635e148f589136260fbf424f5" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 45591 } }
I20260808 18:22:53.955494 29950 raft_consensus.cc:374] T 00000000000000000000000000000000 P 263eef1635e148f589136260fbf424f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:53.955588 29950 raft_consensus.cc:729] T 00000000000000000000000000000000 P 263eef1635e148f589136260fbf424f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 263eef1635e148f589136260fbf424f5, State: Initialized, Role: FOLLOWER
I20260808 18:22:53.956104 29950 consensus_queue.cc:260] T 00000000000000000000000000000000 P 263eef1635e148f589136260fbf424f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "263eef1635e148f589136260fbf424f5" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 45591 } }
I20260808 18:22:53.956257 29950 raft_consensus.cc:388] T 00000000000000000000000000000000 P 263eef1635e148f589136260fbf424f5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:22:53.956357 29950 raft_consensus.cc:482] T 00000000000000000000000000000000 P 263eef1635e148f589136260fbf424f5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:22:53.956455 29950 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 263eef1635e148f589136260fbf424f5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:53.957360 29950 raft_consensus.cc:504] T 00000000000000000000000000000000 P 263eef1635e148f589136260fbf424f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "263eef1635e148f589136260fbf424f5" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 45591 } }
I20260808 18:22:53.957707 29950 leader_election.cc:302] T 00000000000000000000000000000000 P 263eef1635e148f589136260fbf424f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 263eef1635e148f589136260fbf424f5; no voters:
I20260808 18:22:53.958067 29950 leader_election.cc:288] T 00000000000000000000000000000000 P 263eef1635e148f589136260fbf424f5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:22:53.958155 29956 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 263eef1635e148f589136260fbf424f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:22:53.958338 29956 raft_consensus.cc:686] T 00000000000000000000000000000000 P 263eef1635e148f589136260fbf424f5 [term 1 LEADER]: Becoming Leader. State: Replica: 263eef1635e148f589136260fbf424f5, State: Running, Role: LEADER
I20260808 18:22:53.958700 29956 consensus_queue.cc:237] T 00000000000000000000000000000000 P 263eef1635e148f589136260fbf424f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "263eef1635e148f589136260fbf424f5" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 45591 } }
I20260808 18:22:53.959008 29953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:53.944702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58840 (local address 127.23.72.190:45591)
0808 18:22:53.945161 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:53.945176 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:53.945213 (+ 37us) server_negotiation.cc:407] Connection header received
0808 18:22:53.945848 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:53.945865 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:53.946182 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:53.946547 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:53.947508 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:53.948588 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:53.949827 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:53.950350 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:53.950546 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:53.953230 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:53.953235 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:53.953248 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:53.953277 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:53.955718 (+ 2441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:53.956210 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:53.956216 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:53.956222 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:53.956407 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:53.956711 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:53.956716 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:53.956718 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:53.957123 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:53.957343 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:53.957634 (+ 291us) server_negotiation.cc:299] Negotiation successful
0808 18:22:53.957982 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":147,"threads_started":1}
I20260808 18:22:53.959357 29950 sys_catalog.cc:564] T 00000000000000000000000000000000 P 263eef1635e148f589136260fbf424f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:22:53.960366 29957 sys_catalog.cc:455] T 00000000000000000000000000000000 P 263eef1635e148f589136260fbf424f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "263eef1635e148f589136260fbf424f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "263eef1635e148f589136260fbf424f5" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 45591 } } }
I20260808 18:22:53.960489 29957 sys_catalog.cc:458] T 00000000000000000000000000000000 P 263eef1635e148f589136260fbf424f5 [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:53.960826 29958 sys_catalog.cc:455] T 00000000000000000000000000000000 P 263eef1635e148f589136260fbf424f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 263eef1635e148f589136260fbf424f5. Latest consensus state: current_term: 1 leader_uuid: "263eef1635e148f589136260fbf424f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "263eef1635e148f589136260fbf424f5" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 45591 } } }
I20260808 18:22:53.960904 29958 sys_catalog.cc:458] T 00000000000000000000000000000000 P 263eef1635e148f589136260fbf424f5 [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:53.961361 29960 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:22:53.964457 29960 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:22:53.970657 29960 catalog_manager.cc:1347] Generated new cluster ID: 3603cf15f52a43939588dd9fce623c5c
I20260808 18:22:53.970746 29960 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:22:53.989693 29960 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:22:53.991003 29960 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:22:54.001137 29960 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 263eef1635e148f589136260fbf424f5: Generated new TSK 0
I20260808 18:22:54.001969 29960 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.72.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab.
Entry for principal kudu/127.23.72.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab.
I20260808 18:22:54.039405 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.129:0
--local_ip_for_outbound_sockets=127.23.72.129
--webserver_interface=127.23.72.129
--webserver_port=0
--tserver_master_addrs=127.23.72.190:45591
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab
--principal=kudu/127.23.72.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:44187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:54.157162 29977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:54.157435 29977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:54.157503 29977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:54.161602 29977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:54.161720 29977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:54.161839 29977 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.129
I20260808 18:22:54.167093 29977 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:44187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab
--principal=kudu/127.23.72.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.72.129
--webserver_port=0
--tserver_master_addrs=127.23.72.190:45591
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29977
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.129
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:54.168154 29977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:54.169171 29977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:54.178710 29986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:54.178710 29982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:54.178710 29983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:54.179178 29977 server_base.cc:1034] running on GCE node
I20260808 18:22:54.179649 29977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:54.180389 29977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:54.181596 29977 hybrid_clock.cc:648] HybridClock initialized: now 1786213374181579 us; error 35 us; skew 500 ppm
Aug 08 18:22:54 dist-test-slave-5k54 krb5kdc[29864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213374, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:54.184628 29977 init.cc:377] Logged in from keytab as kudu/127.23.72.129@KRBTEST.COM (short username kudu)
I20260808 18:22:54.185547 29977 webserver.cc:460] Webserver started at http://127.23.72.129:37931/ using document root <none> and password file <none>
I20260808 18:22:54.186136 29977 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:54.186241 29977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:54.186486 29977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:54.188577 29977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-0/data/instance:
uuid: "425f755dfdf64f559381d1981229e1cf"
format_stamp: "Formatted at 2026-08-08 18:22:54 on dist-test-slave-5k54"
server_key: "1cb82adf946db5dad6774107d5452138"
server_key_iv: "a5ad4004deddd7b64dda9c9f33576898"
server_key_version: "encryptionkey@0"
I20260808 18:22:54.189169 29977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance:
uuid: "425f755dfdf64f559381d1981229e1cf"
format_stamp: "Formatted at 2026-08-08 18:22:54 on dist-test-slave-5k54"
server_key: "1cb82adf946db5dad6774107d5452138"
server_key_iv: "a5ad4004deddd7b64dda9c9f33576898"
server_key_version: "encryptionkey@0"
I20260808 18:22:54.192924 29977 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:22:54.195307 29993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:54.196524 29977 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 18:22:54.196660 29977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-0/wal
uuid: "425f755dfdf64f559381d1981229e1cf"
format_stamp: "Formatted at 2026-08-08 18:22:54 on dist-test-slave-5k54"
server_key: "1cb82adf946db5dad6774107d5452138"
server_key_iv: "a5ad4004deddd7b64dda9c9f33576898"
server_key_version: "encryptionkey@0"
I20260808 18:22:54.196766 29977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:54.210363 29977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:54.213214 29977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:54.213415 29977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:54.213967 29977 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:22:54.214849 29977 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:22:54.214918 29977 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:54.214984 29977 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:22:54.215037 29977 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:54.224211 29977 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.129:34813
I20260808 18:22:54.224243 30115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.129:34813 every 8 connection(s)
I20260808 18:22:54.225090 29977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
I20260808 18:22:54.227758 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 29977
I20260808 18:22:54.227898 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance
I20260808 18:22:54.228231 23842 external_mini_cluster.cc:1428] Setting key 369200f5be479ff0fc5d6b2dff6f0b12
Aug 08 18:22:54 dist-test-slave-5k54 krb5kdc[29864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213374, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.72.130@KRBTEST.COM; defaulting to no policy
I20260808 18:22:54.240626 29953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:54.226983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:40021 (local address 127.23.72.190:45591)
0808 18:22:54.227126 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:54.227129 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:54.228213 (+ 1084us) server_negotiation.cc:407] Connection header received
0808 18:22:54.229921 (+ 1708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:54.229925 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:54.229976 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:54.230068 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:54.231289 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:54.232197 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:54.233823 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:54.234063 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:54.234182 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:54.236749 (+ 2567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:54.236753 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:54.236755 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:54.236792 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:54.238388 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:54.238930 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:54.238934 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:54.238936 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:54.239027 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:54.239396 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:54.239400 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:54.239402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:54.239620 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:54.239734 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:54.240310 (+ 576us) server_negotiation.cc:299] Negotiation successful
0808 18:22:54.240448 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 18:22:54.241490 30119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:54.227386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:45591 (local address 127.23.72.129:40021)
0808 18:22:54.228037 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:54.228093 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:54.229675 (+ 1582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:54.230301 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:54.230308 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:54.230663 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:54.231128 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:54.231141 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:54.232351 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:54.232356 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:54.233688 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:54.233698 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:54.234224 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:54.234228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:54.234407 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:54.235065 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:54.235081 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:54.236577 (+ 1496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:54.238522 (+ 1945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:54.238529 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:54.238543 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:54.238816 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:54.239129 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:54.239133 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:54.239135 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:54.239292 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:54.239733 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:54.239740 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:54.240062 (+ 322us) client_negotiation.cc:769] Sending connection context
0808 18:22:54.240253 (+ 191us) client_negotiation.cc:240] Negotiation successful
0808 18:22:54.240578 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":146,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.130@KRBTEST.COM" created.
I20260808 18:22:54.242799 30117 heartbeater.cc:347] Connected to a master server at 127.23.72.190:45591
I20260808 18:22:54.243068 30117 heartbeater.cc:464] Registering TS with master...
I20260808 18:22:54.243626 30117 heartbeater.cc:511] Master 127.23.72.190:45591 requested a full tablet report, sending...
I20260808 18:22:54.245244 29911 ts_manager.cc:194] Registered new tserver with Master: 425f755dfdf64f559381d1981229e1cf (127.23.72.129:34813)
I20260808 18:22:54.246932 29911 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.129@KRBTEST.COM'} at 127.23.72.129:40021
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab.
Entry for principal kudu/127.23.72.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab.
I20260808 18:22:54.258196 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.130:0
--local_ip_for_outbound_sockets=127.23.72.130
--webserver_interface=127.23.72.130
--webserver_port=0
--tserver_master_addrs=127.23.72.190:45591
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab
--principal=kudu/127.23.72.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:44187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:54.369089 30124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:54.369330 30124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:54.369369 30124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:54.373374 30124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:54.373426 30124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:54.373526 30124 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.130
I20260808 18:22:54.378526 30124 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:44187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab
--principal=kudu/127.23.72.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.72.130
--webserver_port=0
--tserver_master_addrs=127.23.72.190:45591
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30124
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.130
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:54.379518 30124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:54.380484 30124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:54.389870 30132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:54.389905 30130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:54.389905 30129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:54.390156 30124 server_base.cc:1034] running on GCE node
I20260808 18:22:54.390570 30124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:54.391208 30124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:54.392398 30124 hybrid_clock.cc:648] HybridClock initialized: now 1786213374392377 us; error 42 us; skew 500 ppm
Aug 08 18:22:54 dist-test-slave-5k54 krb5kdc[29864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213374, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:54.395145 30124 init.cc:377] Logged in from keytab as kudu/127.23.72.130@KRBTEST.COM (short username kudu)
I20260808 18:22:54.396060 30124 webserver.cc:460] Webserver started at http://127.23.72.130:41229/ using document root <none> and password file <none>
I20260808 18:22:54.396603 30124 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:54.396692 30124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:54.396925 30124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:54.398909 30124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-1/data/instance:
uuid: "d51fffe697e04ce8bca4a0efef16ce63"
format_stamp: "Formatted at 2026-08-08 18:22:54 on dist-test-slave-5k54"
server_key: "b5b2e4dc5324424c16309b25af889b70"
server_key_iv: "163abc227ad03a9fd21a815f34bb6081"
server_key_version: "encryptionkey@0"
I20260808 18:22:54.399523 30124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance:
uuid: "d51fffe697e04ce8bca4a0efef16ce63"
format_stamp: "Formatted at 2026-08-08 18:22:54 on dist-test-slave-5k54"
server_key: "b5b2e4dc5324424c16309b25af889b70"
server_key_iv: "163abc227ad03a9fd21a815f34bb6081"
server_key_version: "encryptionkey@0"
I20260808 18:22:54.403280 30124 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 18:22:54.405673 30140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:54.406860 30124 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:22:54.406994 30124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-1/wal
uuid: "d51fffe697e04ce8bca4a0efef16ce63"
format_stamp: "Formatted at 2026-08-08 18:22:54 on dist-test-slave-5k54"
server_key: "b5b2e4dc5324424c16309b25af889b70"
server_key_iv: "163abc227ad03a9fd21a815f34bb6081"
server_key_version: "encryptionkey@0"
I20260808 18:22:54.407099 30124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:54.421428 30124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:54.424466 30124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:54.424659 30124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:54.425240 30124 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:22:54.426079 30124 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:22:54.426149 30124 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:54.426221 30124 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:22:54.426273 30124 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:54.435374 30124 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.130:42975
I20260808 18:22:54.435398 30264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.130:42975 every 8 connection(s)
I20260808 18:22:54.436357 30124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
I20260808 18:22:54.445015 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 30124
I20260808 18:22:54.445192 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance
I20260808 18:22:54.445528 23842 external_mini_cluster.cc:1428] Setting key 9f98cef6790e68663c1ab10f85a2b15a
Aug 08 18:22:54 dist-test-slave-5k54 krb5kdc[29864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213374, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.130@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:22:54.450541 29953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:54.438190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:52255 (local address 127.23.72.190:45591)
0808 18:22:54.438341 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:54.438344 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:54.439184 (+ 840us) server_negotiation.cc:407] Connection header received
0808 18:22:54.440552 (+ 1368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:54.440555 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:54.440614 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:54.440722 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:54.441841 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:54.442376 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:54.443530 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:54.443763 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:54.443875 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:54.446470 (+ 2595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:54.446474 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:54.446477 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:54.446508 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:54.448360 (+ 1852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:54.448863 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:54.448869 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:54.448874 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:54.448974 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:54.449295 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:54.449301 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:54.449305 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:54.449583 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:54.449662 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:54.450293 (+ 631us) server_negotiation.cc:299] Negotiation successful
0808 18:22:54.450419 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 18:22:54.451709 30267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:54.438578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:45591 (local address 127.23.72.130:52255)
0808 18:22:54.439019 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:54.439060 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:54.440324 (+ 1264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:54.440896 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:54.440909 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:54.441249 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:54.441692 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:54.441704 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:54.442517 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:54.442525 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:54.443382 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:54.443391 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:54.443922 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:54.443927 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:54.444091 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:54.444634 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:54.444650 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:54.446303 (+ 1653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:54.448479 (+ 2176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:54.448484 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:54.448495 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:54.448761 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:54.449067 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:54.449069 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:54.449070 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:54.449204 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:54.449691 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:54.449701 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:54.450027 (+ 326us) client_negotiation.cc:769] Sending connection context
0808 18:22:54.450249 (+ 222us) client_negotiation.cc:240] Negotiation successful
0808 18:22:54.450611 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":111,"threads_started":1}
I20260808 18:22:54.452893 30265 heartbeater.cc:347] Connected to a master server at 127.23.72.190:45591
I20260808 18:22:54.453138 30265 heartbeater.cc:464] Registering TS with master...
I20260808 18:22:54.453742 30265 heartbeater.cc:511] Master 127.23.72.190:45591 requested a full tablet report, sending...
I20260808 18:22:54.454931 29911 ts_manager.cc:194] Registered new tserver with Master: d51fffe697e04ce8bca4a0efef16ce63 (127.23.72.130:42975)
I20260808 18:22:54.455588 29911 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.130@KRBTEST.COM'} at 127.23.72.130:52255
WARNING: no policy specified for kudu/127.23.72.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab.
Entry for principal kudu/127.23.72.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab.
I20260808 18:22:54.478782 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.131:0
--local_ip_for_outbound_sockets=127.23.72.131
--webserver_interface=127.23.72.131
--webserver_port=0
--tserver_master_addrs=127.23.72.190:45591
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab
--principal=kudu/127.23.72.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:44187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:54.591044 30270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:54.591284 30270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:54.591325 30270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:54.595292 30270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:54.595343 30270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:54.595403 30270 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.131
I20260808 18:22:54.600528 30270 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:44187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab
--principal=kudu/127.23.72.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.72.131
--webserver_port=0
--tserver_master_addrs=127.23.72.190:45591
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.131
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:54.601542 30270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:54.602497 30270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:54.611801 30276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:54.611797 30279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:54.611793 30277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:54.612066 30270 server_base.cc:1034] running on GCE node
I20260808 18:22:54.612541 30270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:54.613162 30270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:54.614355 30270 hybrid_clock.cc:648] HybridClock initialized: now 1786213374614332 us; error 35 us; skew 500 ppm
Aug 08 18:22:54 dist-test-slave-5k54 krb5kdc[29864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213374, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:54.617090 30270 init.cc:377] Logged in from keytab as kudu/127.23.72.131@KRBTEST.COM (short username kudu)
I20260808 18:22:54.617875 30270 webserver.cc:460] Webserver started at http://127.23.72.131:38233/ using document root <none> and password file <none>
I20260808 18:22:54.618408 30270 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:54.618517 30270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:54.618746 30270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:54.620869 30270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-2/data/instance:
uuid: "3ac59582534d4ff690a22808ba710f2d"
format_stamp: "Formatted at 2026-08-08 18:22:54 on dist-test-slave-5k54"
server_key: "9e6f9e542ed3549858924314cc2b6662"
server_key_iv: "c091362ab47114e29985a20e65375c39"
server_key_version: "encryptionkey@0"
I20260808 18:22:54.621444 30270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance:
uuid: "3ac59582534d4ff690a22808ba710f2d"
format_stamp: "Formatted at 2026-08-08 18:22:54 on dist-test-slave-5k54"
server_key: "9e6f9e542ed3549858924314cc2b6662"
server_key_iv: "c091362ab47114e29985a20e65375c39"
server_key_version: "encryptionkey@0"
I20260808 18:22:54.625074 30270 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 18:22:54.627408 30286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:54.628587 30270 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:22:54.628718 30270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-2/wal
uuid: "3ac59582534d4ff690a22808ba710f2d"
format_stamp: "Formatted at 2026-08-08 18:22:54 on dist-test-slave-5k54"
server_key: "9e6f9e542ed3549858924314cc2b6662"
server_key_iv: "c091362ab47114e29985a20e65375c39"
server_key_version: "encryptionkey@0"
I20260808 18:22:54.628827 30270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:54.637672 30270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:54.640796 30270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:54.640990 30270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:54.641556 30270 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:22:54.642314 30270 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:22:54.642385 30270 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:54.642463 30270 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:22:54.642515 30270 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:54.651134 30270 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.131:33449
I20260808 18:22:54.651204 30412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.131:33449 every 8 connection(s)
I20260808 18:22:54.652010 30270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
I20260808 18:22:54.655354 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 30270
I20260808 18:22:54.655524 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance
I20260808 18:22:54.655858 23842 external_mini_cluster.cc:1428] Setting key b445b47e04f97eb272b8693ee6014c48
Aug 08 18:22:54 dist-test-slave-5k54 krb5kdc[29864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213374, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.131@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:22:54.665520 29953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:54.653705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:32853 (local address 127.23.72.190:45591)
0808 18:22:54.653849 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:54.653853 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:54.654639 (+ 786us) server_negotiation.cc:407] Connection header received
0808 18:22:54.656373 (+ 1734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:54.656377 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:54.656435 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:54.656545 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:54.657770 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:54.658346 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:54.659557 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:54.659784 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:54.659911 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:54.662232 (+ 2321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:54.662236 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:54.662239 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:54.662269 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:54.663547 (+ 1278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:54.664045 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:54.664049 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:54.664051 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:54.664134 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:54.664479 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:54.664482 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:54.664484 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:54.664659 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:54.664778 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:54.665238 (+ 460us) server_negotiation.cc:299] Negotiation successful
0808 18:22:54.665358 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 18:22:54.666246 30415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:54.654053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:45591 (local address 127.23.72.131:32853)
0808 18:22:54.654498 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:54.654538 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:54.656148 (+ 1610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:54.656708 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:54.656714 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:54.657167 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:54.657630 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:54.657643 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:54.658470 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:54.658474 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:54.659394 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:54.659403 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:54.659941 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:54.659945 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:54.660114 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:54.660624 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:54.660640 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:54.662080 (+ 1440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:54.663665 (+ 1585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:54.663671 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:54.663684 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:54.663939 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:54.664227 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:54.664231 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:54.664233 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:54.664367 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:54.664775 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:54.664781 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:54.665011 (+ 230us) client_negotiation.cc:769] Sending connection context
0808 18:22:54.665195 (+ 184us) client_negotiation.cc:240] Negotiation successful
0808 18:22:54.665506 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":108,"threads_started":1}
I20260808 18:22:54.667275 30413 heartbeater.cc:347] Connected to a master server at 127.23.72.190:45591
I20260808 18:22:54.667562 30413 heartbeater.cc:464] Registering TS with master...
I20260808 18:22:54.667995 30413 heartbeater.cc:511] Master 127.23.72.190:45591 requested a full tablet report, sending...
I20260808 18:22:54.669034 29911 ts_manager.cc:194] Registered new tserver with Master: 3ac59582534d4ff690a22808ba710f2d (127.23.72.131:33449)
I20260808 18:22:54.669651 23842 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:22:54.669688 29911 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.131@KRBTEST.COM'} at 127.23.72.131:32853
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/krb5kdc/test-admin.keytab.
Aug 08 18:22:54 dist-test-slave-5k54 krb5kdc[29864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:54.688958 23842 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260808 18:22:54.691761 30420 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:22:54.690297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47364 (local address 127.23.72.129:34813)
0808 18:22:54.690604 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:54.690620 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:54.690643 (+ 23us) server_negotiation.cc:407] Connection header received
0808 18:22:54.690692 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:54.690697 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:54.690871 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:54.691010 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:54.691562 (+ 552us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47364: BlockingRecv error: recv got EOF from 127.0.0.1:47364 (error 108)
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":124,"threads_started":1}
W20260808 18:22:54.691780 23842 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/krb5kdc/krb5cc' of type FILE
Aug 08 18:22:54 dist-test-slave-5k54 krb5kdc[29864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:54.692812 23842 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 18:22:54 dist-test-slave-5k54 krb5kdc[29864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:54.693737 23842 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 18:22:54 dist-test-slave-5k54 krb5kdc[29864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.72.129@KRBTEST.COM
I20260808 18:22:54.703469 30420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:54.694570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47378 (local address 127.23.72.129:34813)
0808 18:22:54.694733 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:54.694738 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:54.694830 (+ 92us) server_negotiation.cc:407] Connection header received
0808 18:22:54.694922 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:54.694925 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:54.694963 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:54.695031 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:54.695593 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:54.696460 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:54.697239 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:54.697619 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:54.697719 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:54.700345 (+ 2626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:54.700355 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:54.700367 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:54.700389 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:54.701875 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:54.702255 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:54.702259 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:54.702260 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:54.702335 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:54.702556 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:54.702558 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:54.702560 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:54.702775 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:54.702954 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:54.703093 (+ 139us) server_negotiation.cc:299] Negotiation successful
0808 18:22:54.703257 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260808 18:22:54.707242 30420 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:22:54.706213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47386 (local address 127.23.72.129:34813)
0808 18:22:54.706353 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:54.706358 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:54.706444 (+ 86us) server_negotiation.cc:407] Connection header received
0808 18:22:54.706598 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:54.706600 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:54.706636 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:54.706721 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:54.707126 (+ 405us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47386: BlockingRecv error: recv got EOF from 127.0.0.1:47386 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260808 18:22:54.707407 23842 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/krb5kdc/krb5cc' of type FILE
Aug 08 18:22:54 dist-test-slave-5k54 krb5kdc[29864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:54.708424 23842 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 18:22:54 dist-test-slave-5k54 krb5kdc[29864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:54.709324 23842 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 18:22:54 dist-test-slave-5k54 krb5kdc[29864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.72.129@KRBTEST.COM
I20260808 18:22:54.717602 30420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:54.710078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47400 (local address 127.23.72.129:34813)
0808 18:22:54.710242 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:54.710247 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:54.710263 (+ 16us) server_negotiation.cc:407] Connection header received
0808 18:22:54.710373 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:54.710375 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:54.710406 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:54.710466 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:54.711015 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:54.711620 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:54.712370 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:54.712577 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:54.712650 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:54.714725 (+ 2075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:54.714728 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:54.714730 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:54.714750 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:54.716213 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:54.716584 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:54.716587 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:54.716588 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:54.716656 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:54.716870 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:54.716873 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:54.716874 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:54.717031 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:54.717123 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:54.717322 (+ 199us) server_negotiation.cc:299] Negotiation successful
0808 18:22:54.717443 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260808 18:22:54.720180 30420 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:22:54.719277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47402 (local address 127.23.72.129:34813)
0808 18:22:54.719406 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:54.719409 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:54.719461 (+ 52us) server_negotiation.cc:407] Connection header received
0808 18:22:54.719631 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:54.719633 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:54.719665 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:54.719726 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:54.720088 (+ 362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47402: BlockingRecv error: recv got EOF from 127.0.0.1:47402 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260808 18:22:54.720368 23842 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213345431840-23842-0/krb5kdc/krb5cc' of type FILE
Aug 08 18:22:54 dist-test-slave-5k54 krb5kdc[29864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:54.721340 23842 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 18:22:54 dist-test-slave-5k54 krb5kdc[29864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:54.722241 23842 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 18:22:54 dist-test-slave-5k54 krb5kdc[29864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.72.129@KRBTEST.COM
I20260808 18:22:54.730835 30420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:54.723113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47416 (local address 127.23.72.129:34813)
0808 18:22:54.723239 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:54.723242 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:54.723333 (+ 91us) server_negotiation.cc:407] Connection header received
0808 18:22:54.723400 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:54.723402 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:54.723496 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:54.723578 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:54.724129 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:54.724622 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:54.725529 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:54.725732 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:54.725820 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:54.728083 (+ 2263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:54.728086 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:54.728087 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:54.728109 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:54.729437 (+ 1328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:54.729823 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:54.729826 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:54.729827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:54.729895 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:54.730120 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:54.730123 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:54.730124 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:54.730261 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:54.730348 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:54.730537 (+ 189us) server_negotiation.cc:299] Negotiation successful
0808 18:22:54.730651 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 18:22:54.731887 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 29977
I20260808 18:22:54.740062 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 30124
I20260808 18:22:54.747939 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 30270
I20260808 18:22:54.754375 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 29878
2026-08-08T18:22:54Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2231 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:22:54 dist-test-slave-5k54 krb5kdc[30435](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:22:54 dist-test-slave-5k54 krb5kdc[30435](info): set up 2 sockets
Aug 08 18:22:54 dist-test-slave-5k54 krb5kdc[30435](info): commencing operation
krb5kdc: starting...
W20260808 18:22:55.861220 23842 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Aug 08 18:22:55 dist-test-slave-5k54 krb5kdc[30435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:22:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:22:55Z Disabled control of system clock
WARNING: no policy specified for oryx/127.23.72.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.72.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.72.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.190.keytab.
Entry for principal oryx/127.23.72.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.190.keytab.
I20260808 18:22:55.986501 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:36587
--webserver_interface=127.23.72.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.190.keytab
--principal=oryx/127.23.72.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:35011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:36587
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:56.098059 30448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:56.098337 30448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:56.098378 30448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:56.102337 30448 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:22:56.102394 30448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:56.102413 30448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:56.102429 30448 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:22:56.102444 30448 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:22:56.107633 30448 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:35011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:36587
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.190.keytab
--principal=oryx/127.23.72.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:36587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.72.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30448
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:56.108721 30448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:56.109711 30448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:56.118145 30457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:56.118202 30454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:56.118171 30455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:56.118427 30448 server_base.cc:1034] running on GCE node
I20260808 18:22:56.119163 30448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:56.120424 30448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:56.121625 30448 hybrid_clock.cc:648] HybridClock initialized: now 1786213376121608 us; error 32 us; skew 500 ppm
Aug 08 18:22:56 dist-test-slave-5k54 krb5kdc[30435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213376, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.72.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:56.128212 30448 init.cc:377] Logged in from keytab as oryx/127.23.72.190@KRBTEST.COM (short username oryx)
I20260808 18:22:56.129184 30448 webserver.cc:460] Webserver started at http://127.23.72.190:33675/ using document root <none> and password file <none>
I20260808 18:22:56.129770 30448 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:56.129866 30448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:56.130080 30448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:56.132234 30448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18"
format_stamp: "Formatted at 2026-08-08 18:22:56 on dist-test-slave-5k54"
server_key: "7ec1906d94e260f3166d93d82393b4ee"
server_key_iv: "d10c1ab08a3f9fbb9405d1440937865e"
server_key_version: "encryptionkey@0"
I20260808 18:22:56.132840 30448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18"
format_stamp: "Formatted at 2026-08-08 18:22:56 on dist-test-slave-5k54"
server_key: "7ec1906d94e260f3166d93d82393b4ee"
server_key_iv: "d10c1ab08a3f9fbb9405d1440937865e"
server_key_version: "encryptionkey@0"
I20260808 18:22:56.136646 30448 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260808 18:22:56.139170 30465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:56.140426 30448 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 18:22:56.140569 30448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18"
format_stamp: "Formatted at 2026-08-08 18:22:56 on dist-test-slave-5k54"
server_key: "7ec1906d94e260f3166d93d82393b4ee"
server_key_iv: "d10c1ab08a3f9fbb9405d1440937865e"
server_key_version: "encryptionkey@0"
I20260808 18:22:56.140684 30448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:56.179582 30448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:56.182485 30448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:56.182709 30448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:56.189898 30448 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.190:36587
I20260808 18:22:56.190188 30522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.190:36587 every 8 connection(s)
I20260808 18:22:56.190920 30448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:22:56.193243 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 30448
I20260808 18:22:56.193383 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:22:56.193708 23842 external_mini_cluster.cc:1428] Setting key 54ebba47bec84ad93c47b9f209b99ec4
I20260808 18:22:56.194010 30523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:56.201164 30523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18: Bootstrap starting.
Aug 08 18:22:56 dist-test-slave-5k54 krb5kdc[30435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM
I20260808 18:22:56.203737 30523 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:56.204592 30523 log.cc:824] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:56.206786 30523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18: No bootstrap required, opened a new log
I20260808 18:22:56.208807 30526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:56.195240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49354 (local address 127.23.72.190:36587)
0808 18:22:56.195700 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:56.195719 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:56.195761 (+ 42us) server_negotiation.cc:407] Connection header received
0808 18:22:56.196741 (+ 980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:56.196765 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:56.197190 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:56.197556 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:56.198229 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:56.199775 (+ 1546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:56.200609 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:56.201291 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:56.201444 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:56.204159 (+ 2715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:56.204163 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:56.204176 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:56.204204 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:56.205977 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:56.206448 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:56.206452 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:56.206456 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:56.206558 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:56.206892 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:56.206897 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:56.206903 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:56.207257 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:56.207397 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:56.207727 (+ 330us) server_negotiation.cc:299] Negotiation successful
0808 18:22:56.208056 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":172,"threads_started":1}
I20260808 18:22:56.210791 30523 raft_consensus.cc:348] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } }
I20260808 18:22:56.211082 30523 raft_consensus.cc:374] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:56.211155 30523 raft_consensus.cc:729] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9d826ba0df445bfb8cc41e8dc8c9f18, State: Initialized, Role: FOLLOWER
I20260808 18:22:56.211606 30523 consensus_queue.cc:260] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } }
I20260808 18:22:56.211728 30523 raft_consensus.cc:388] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:22:56.211773 30523 raft_consensus.cc:482] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:22:56.211843 30523 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:56.212621 30523 raft_consensus.cc:504] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } }
I20260808 18:22:56.212896 30523 leader_election.cc:302] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e9d826ba0df445bfb8cc41e8dc8c9f18; no voters:
I20260808 18:22:56.213148 30523 leader_election.cc:288] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:22:56.213269 30528 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:22:56.213533 30528 raft_consensus.cc:686] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 [term 1 LEADER]: Becoming Leader. State: Replica: e9d826ba0df445bfb8cc41e8dc8c9f18, State: Running, Role: LEADER
I20260808 18:22:56.213871 30528 consensus_queue.cc:237] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } }
I20260808 18:22:56.214349 30523 sys_catalog.cc:564] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:22:56.215668 30529 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } }
I20260808 18:22:56.215797 30529 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:56.216197 30538 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:22:56.216146 30530 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9d826ba0df445bfb8cc41e8dc8c9f18. Latest consensus state: current_term: 1 leader_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } }
I20260808 18:22:56.217372 30530 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:56.219944 30538 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:22:56.225471 30538 catalog_manager.cc:1347] Generated new cluster ID: bf3f1e45a3cf47969ef2d1762c680e0e
I20260808 18:22:56.225539 30538 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:22:56.241957 30538 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:22:56.242865 30538 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:22:56.256345 30538 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18: Generated new TSK 0
I20260808 18:22:56.257122 30538 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.23.72.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.72.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.72.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.129.keytab.
Entry for principal oryx/127.23.72.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.129.keytab.
I20260808 18:22:56.295967 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.129:0
--local_ip_for_outbound_sockets=127.23.72.129
--webserver_interface=127.23.72.129
--webserver_port=0
--tserver_master_addrs=127.23.72.190:36587
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.129.keytab
--principal=oryx/127.23.72.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:35011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:56.409730 30551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:56.409993 30551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:56.410036 30551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:56.414090 30551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:56.414150 30551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:56.414209 30551 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.129
I20260808 18:22:56.419625 30551 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:35011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.129.keytab
--principal=oryx/127.23.72.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.72.129
--webserver_port=0
--tserver_master_addrs=127.23.72.190:36587
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30551
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.129
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:56.420643 30551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:56.421614 30551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:56.431218 30558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:56.431231 30551 server_base.cc:1034] running on GCE node
W20260808 18:22:56.431227 30561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:56.431350 30559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:56.431914 30551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:56.432607 30551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:56.433765 30551 hybrid_clock.cc:648] HybridClock initialized: now 1786213376433738 us; error 40 us; skew 500 ppm
Aug 08 18:22:56 dist-test-slave-5k54 krb5kdc[30435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213376, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.72.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:56.436801 30551 init.cc:377] Logged in from keytab as oryx/127.23.72.129@KRBTEST.COM (short username oryx)
I20260808 18:22:56.437704 30551 webserver.cc:460] Webserver started at http://127.23.72.129:43901/ using document root <none> and password file <none>
I20260808 18:22:56.438220 30551 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:56.438277 30551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:56.438452 30551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:56.440521 30551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-0/data/instance:
uuid: "0c00472449774db482dac0a3c10272ed"
format_stamp: "Formatted at 2026-08-08 18:22:56 on dist-test-slave-5k54"
server_key: "0cb2368f4810873075043f295869d2b5"
server_key_iv: "d49d2250ec3080345c9876cb6a0a5585"
server_key_version: "encryptionkey@0"
I20260808 18:22:56.441072 30551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance:
uuid: "0c00472449774db482dac0a3c10272ed"
format_stamp: "Formatted at 2026-08-08 18:22:56 on dist-test-slave-5k54"
server_key: "0cb2368f4810873075043f295869d2b5"
server_key_iv: "d49d2250ec3080345c9876cb6a0a5585"
server_key_version: "encryptionkey@0"
I20260808 18:22:56.444990 30551 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 18:22:56.447531 30568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:56.448758 30551 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:22:56.448854 30551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-0/wal
uuid: "0c00472449774db482dac0a3c10272ed"
format_stamp: "Formatted at 2026-08-08 18:22:56 on dist-test-slave-5k54"
server_key: "0cb2368f4810873075043f295869d2b5"
server_key_iv: "d49d2250ec3080345c9876cb6a0a5585"
server_key_version: "encryptionkey@0"
I20260808 18:22:56.448946 30551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:56.457970 30551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:56.460757 30551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:56.460922 30551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:56.461431 30551 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:22:56.462167 30551 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:22:56.462212 30551 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:56.462250 30551 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:22:56.462313 30551 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:56.471105 30551 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.129:38285
I20260808 18:22:56.471148 30693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.129:38285 every 8 connection(s)
I20260808 18:22:56.472061 30551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
I20260808 18:22:56.472836 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 30551
I20260808 18:22:56.472980 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance
I20260808 18:22:56.473311 23842 external_mini_cluster.cc:1428] Setting key 26981ca5623aad1a5f2e15037243f89f
Aug 08 18:22:56 dist-test-slave-5k54 krb5kdc[30435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213376, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.72.129@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM
WARNING: no policy specified for oryx/127.23.72.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.72.130@KRBTEST.COM" created.
I20260808 18:22:56.488654 30526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:56.474010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:38281 (local address 127.23.72.190:36587)
0808 18:22:56.474182 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:56.474187 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:56.475230 (+ 1043us) server_negotiation.cc:407] Connection header received
0808 18:22:56.477306 (+ 2076us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:56.477313 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:56.477379 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:56.477483 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:56.478685 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:56.479647 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:56.480892 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:56.481352 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:56.481492 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:56.484322 (+ 2830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:56.484327 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:56.484330 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:56.484361 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:56.486039 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:56.486566 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:56.486569 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:56.486570 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:56.486663 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:56.487178 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:56.487183 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:56.487204 (+ 21us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:56.487484 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:56.487639 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:56.488324 (+ 685us) server_negotiation.cc:299] Negotiation successful
0808 18:22:56.488445 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 18:22:56.489741 30697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:56.474461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:36587 (local address 127.23.72.129:38281)
0808 18:22:56.475050 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:56.475107 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:56.477035 (+ 1928us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:56.477644 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:56.477650 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:56.478038 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:56.478528 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:56.478541 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:56.479778 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:56.479782 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:56.480746 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:56.480756 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:56.481532 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:56.481535 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:56.481682 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:56.482362 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:56.482378 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:56.484135 (+ 1757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:56.486151 (+ 2016us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:56.486157 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:56.486171 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:56.486445 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:56.486755 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:56.486757 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:56.486764 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:56.486958 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:56.487634 (+ 676us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:56.487641 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:56.488006 (+ 365us) client_negotiation.cc:769] Sending connection context
0808 18:22:56.488272 (+ 266us) client_negotiation.cc:240] Negotiation successful
0808 18:22:56.488692 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":147,"threads_started":1}
I20260808 18:22:56.491159 30694 heartbeater.cc:347] Connected to a master server at 127.23.72.190:36587
I20260808 18:22:56.491482 30694 heartbeater.cc:464] Registering TS with master...
I20260808 18:22:56.492026 30694 heartbeater.cc:511] Master 127.23.72.190:36587 requested a full tablet report, sending...
I20260808 18:22:56.493837 30484 ts_manager.cc:194] Registered new tserver with Master: 0c00472449774db482dac0a3c10272ed (127.23.72.129:38285)
I20260808 18:22:56.495625 30484 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.72.129@KRBTEST.COM'} at 127.23.72.129:38281
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.72.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.130.keytab.
Entry for principal oryx/127.23.72.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.130.keytab.
I20260808 18:22:56.506371 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.130:0
--local_ip_for_outbound_sockets=127.23.72.130
--webserver_interface=127.23.72.130
--webserver_port=0
--tserver_master_addrs=127.23.72.190:36587
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.130.keytab
--principal=oryx/127.23.72.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:35011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:56.617941 30701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:56.618189 30701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:56.618232 30701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:56.622187 30701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:56.622248 30701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:56.622303 30701 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.130
I20260808 18:22:56.627360 30701 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:35011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.130.keytab
--principal=oryx/127.23.72.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.72.130
--webserver_port=0
--tserver_master_addrs=127.23.72.190:36587
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.130
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:56.628473 30701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:56.629437 30701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:56.638811 30710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:56.638826 30707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:56.638921 30708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:56.639237 30701 server_base.cc:1034] running on GCE node
I20260808 18:22:56.639683 30701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:56.640295 30701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:56.641481 30701 hybrid_clock.cc:648] HybridClock initialized: now 1786213376641470 us; error 27 us; skew 500 ppm
Aug 08 18:22:56 dist-test-slave-5k54 krb5kdc[30435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213376, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.72.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:56.644340 30701 init.cc:377] Logged in from keytab as oryx/127.23.72.130@KRBTEST.COM (short username oryx)
I20260808 18:22:56.645224 30701 webserver.cc:460] Webserver started at http://127.23.72.130:34621/ using document root <none> and password file <none>
I20260808 18:22:56.645740 30701 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:56.645798 30701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:56.645972 30701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:56.648046 30701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-1/data/instance:
uuid: "f9977432298448988bdb259803c1171b"
format_stamp: "Formatted at 2026-08-08 18:22:56 on dist-test-slave-5k54"
server_key: "3f4bd2ba953dc8accdcaddfaa0beac0c"
server_key_iv: "7f6301dae52f6361552eab98d98dfe31"
server_key_version: "encryptionkey@0"
I20260808 18:22:56.648581 30701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance:
uuid: "f9977432298448988bdb259803c1171b"
format_stamp: "Formatted at 2026-08-08 18:22:56 on dist-test-slave-5k54"
server_key: "3f4bd2ba953dc8accdcaddfaa0beac0c"
server_key_iv: "7f6301dae52f6361552eab98d98dfe31"
server_key_version: "encryptionkey@0"
I20260808 18:22:56.652297 30701 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:22:56.654753 30718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:56.656091 30701 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:22:56.656217 30701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-1/wal
uuid: "f9977432298448988bdb259803c1171b"
format_stamp: "Formatted at 2026-08-08 18:22:56 on dist-test-slave-5k54"
server_key: "3f4bd2ba953dc8accdcaddfaa0beac0c"
server_key_iv: "7f6301dae52f6361552eab98d98dfe31"
server_key_version: "encryptionkey@0"
I20260808 18:22:56.656354 30701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:56.673408 30701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:56.676421 30701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:56.676623 30701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:56.677215 30701 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:22:56.678061 30701 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:22:56.678107 30701 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:56.678174 30701 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:22:56.678213 30701 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:56.687165 30701 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.130:33107
I20260808 18:22:56.687235 30843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.130:33107 every 8 connection(s)
I20260808 18:22:56.688134 30701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
I20260808 18:22:56.693394 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 30701
I20260808 18:22:56.693542 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance
I20260808 18:22:56.693913 23842 external_mini_cluster.cc:1428] Setting key 1561f890bf17e286e7e0f7d08a948626
Aug 08 18:22:56 dist-test-slave-5k54 krb5kdc[30435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213376, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.72.130@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM
I20260808 18:22:56.702418 30526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:56.689971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:53483 (local address 127.23.72.190:36587)
0808 18:22:56.690119 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:56.690124 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:56.690945 (+ 821us) server_negotiation.cc:407] Connection header received
0808 18:22:56.692368 (+ 1423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:56.692372 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:56.692433 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:56.692545 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:56.693773 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:56.694377 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:56.695797 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:56.696031 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:56.696142 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:56.698723 (+ 2581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:56.698727 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:56.698730 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:56.698761 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:56.700396 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:56.700897 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:56.700902 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:56.700904 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:56.700989 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:56.701339 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:56.701342 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:56.701344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:56.701527 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:56.701629 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:56.702127 (+ 498us) server_negotiation.cc:299] Negotiation successful
0808 18:22:56.702249 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 18:22:56.703222 30846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:56.690332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:36587 (local address 127.23.72.130:53483)
0808 18:22:56.690777 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:56.690819 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:56.692147 (+ 1328us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:56.692712 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:56.692719 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:56.693068 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:56.693622 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:56.693647 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:56.694503 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:56.694506 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:56.695676 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:56.695685 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:56.696181 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:56.696184 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:56.696350 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:56.697014 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:56.697029 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:56.698556 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:56.700512 (+ 1956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:56.700518 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:56.700531 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:56.700793 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:56.701079 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:56.701083 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:56.701085 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:56.701222 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:56.701641 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:56.701650 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:56.701887 (+ 237us) client_negotiation.cc:769] Sending connection context
0808 18:22:56.702065 (+ 178us) client_negotiation.cc:240] Negotiation successful
0808 18:22:56.702390 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":115,"threads_started":1}
WARNING: no policy specified for oryx/127.23.72.131@KRBTEST.COM; defaulting to no policy
I20260808 18:22:56.704420 30844 heartbeater.cc:347] Connected to a master server at 127.23.72.190:36587
I20260808 18:22:56.704619 30844 heartbeater.cc:464] Registering TS with master...
I20260808 18:22:56.705096 30844 heartbeater.cc:511] Master 127.23.72.190:36587 requested a full tablet report, sending...
I20260808 18:22:56.706173 30484 ts_manager.cc:194] Registered new tserver with Master: f9977432298448988bdb259803c1171b (127.23.72.130:33107)
I20260808 18:22:56.706746 30484 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.72.130@KRBTEST.COM'} at 127.23.72.130:53483
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.72.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.72.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.131.keytab.
Entry for principal oryx/127.23.72.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.131.keytab.
I20260808 18:22:56.726357 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.131:0
--local_ip_for_outbound_sockets=127.23.72.131
--webserver_interface=127.23.72.131
--webserver_port=0
--tserver_master_addrs=127.23.72.190:36587
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.131.keytab
--principal=oryx/127.23.72.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:35011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:56.839469 30850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:56.839738 30850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:56.839782 30850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:56.843824 30850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:56.843881 30850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:56.843941 30850 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.131
I20260808 18:22:56.849153 30850 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:35011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.131.keytab
--principal=oryx/127.23.72.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.72.131
--webserver_port=0
--tserver_master_addrs=127.23.72.190:36587
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.131
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:56.850112 30850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:56.851068 30850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:56.860785 30856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:56.860801 30855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:56.860834 30858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:56.861135 30850 server_base.cc:1034] running on GCE node
I20260808 18:22:56.861784 30850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:56.862450 30850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:56.863664 30850 hybrid_clock.cc:648] HybridClock initialized: now 1786213376863608 us; error 46 us; skew 500 ppm
Aug 08 18:22:56 dist-test-slave-5k54 krb5kdc[30435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213376, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.72.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:56.867039 30850 init.cc:377] Logged in from keytab as oryx/127.23.72.131@KRBTEST.COM (short username oryx)
I20260808 18:22:56.867921 30850 webserver.cc:460] Webserver started at http://127.23.72.131:43467/ using document root <none> and password file <none>
I20260808 18:22:56.868482 30850 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:56.868571 30850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:56.868785 30850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:56.871970 30850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-2/data/instance:
uuid: "086e23e953ee4ecdb9a68942c3037fd5"
format_stamp: "Formatted at 2026-08-08 18:22:56 on dist-test-slave-5k54"
server_key: "6a397a7d5f8fc29cccf95a7878c18637"
server_key_iv: "4a60a8358a03b0ef09fa66178526e8db"
server_key_version: "encryptionkey@0"
I20260808 18:22:56.872779 30850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance:
uuid: "086e23e953ee4ecdb9a68942c3037fd5"
format_stamp: "Formatted at 2026-08-08 18:22:56 on dist-test-slave-5k54"
server_key: "6a397a7d5f8fc29cccf95a7878c18637"
server_key_iv: "4a60a8358a03b0ef09fa66178526e8db"
server_key_version: "encryptionkey@0"
I20260808 18:22:56.877835 30850 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260808 18:22:56.880239 30866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:56.881457 30850 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:22:56.881558 30850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-2/wal
uuid: "086e23e953ee4ecdb9a68942c3037fd5"
format_stamp: "Formatted at 2026-08-08 18:22:56 on dist-test-slave-5k54"
server_key: "6a397a7d5f8fc29cccf95a7878c18637"
server_key_iv: "4a60a8358a03b0ef09fa66178526e8db"
server_key_version: "encryptionkey@0"
I20260808 18:22:56.881695 30850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:56.927573 30850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:56.931021 30850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:56.931241 30850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:56.931900 30850 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:22:56.932739 30850 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:22:56.932785 30850 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:56.932853 30850 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:22:56.932895 30850 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:56.942243 30850 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.131:41097
I20260808 18:22:56.942276 30992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.131:41097 every 8 connection(s)
I20260808 18:22:56.943135 30850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
I20260808 18:22:56.943707 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 30850
I20260808 18:22:56.943859 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance
I20260808 18:22:56.944168 23842 external_mini_cluster.cc:1428] Setting key 4013505775a5e8b6e6d3705252ebac1d
Aug 08 18:22:56 dist-test-slave-5k54 krb5kdc[30435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213376, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.72.131@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM
I20260808 18:22:56.957348 30526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:56.945302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:54041 (local address 127.23.72.190:36587)
0808 18:22:56.945437 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:56.945448 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:56.946481 (+ 1033us) server_negotiation.cc:407] Connection header received
0808 18:22:56.948024 (+ 1543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:56.948027 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:56.948080 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:56.948188 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:56.949366 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:56.949897 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:56.951025 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:56.951246 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:56.951362 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:56.953916 (+ 2554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:56.953927 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:56.953929 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:56.953952 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:56.955171 (+ 1219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:56.955745 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:56.955749 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:56.955751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:56.955836 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:56.956174 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:56.956177 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:56.956179 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:56.956359 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:56.956468 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:56.957025 (+ 557us) server_negotiation.cc:299] Negotiation successful
0808 18:22:56.957157 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 18:22:56.958211 30996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:56.945820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:36587 (local address 127.23.72.131:54041)
0808 18:22:56.946335 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:56.946393 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:56.947828 (+ 1435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:56.948360 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:56.948367 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:56.948714 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:56.949225 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:56.949238 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:56.950027 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:56.950030 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:56.950894 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:56.950908 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:56.951406 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:56.951410 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:56.951593 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:56.952182 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:56.952198 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:56.953722 (+ 1524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:56.955287 (+ 1565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:56.955299 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:56.955317 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:56.955642 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:56.955931 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:56.955935 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:56.955937 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:56.956071 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:56.956465 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:56.956471 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:56.956707 (+ 236us) client_negotiation.cc:769] Sending connection context
0808 18:22:56.956916 (+ 209us) client_negotiation.cc:240] Negotiation successful
0808 18:22:56.957240 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":123,"threads_started":1}
I20260808 18:22:56.959297 30993 heartbeater.cc:347] Connected to a master server at 127.23.72.190:36587
I20260808 18:22:56.959586 30993 heartbeater.cc:464] Registering TS with master...
I20260808 18:22:56.960058 30993 heartbeater.cc:511] Master 127.23.72.190:36587 requested a full tablet report, sending...
I20260808 18:22:56.961045 30484 ts_manager.cc:194] Registered new tserver with Master: 086e23e953ee4ecdb9a68942c3037fd5 (127.23.72.131:41097)
I20260808 18:22:56.961566 30484 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.72.131@KRBTEST.COM'} at 127.23.72.131:54041
I20260808 18:22:56.969786 23842 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.23.72.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.72.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.72.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.189.keytab.
Entry for principal oryx/127.23.72.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.189.keytab.
I20260808 18:22:56.999392 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.189:38137
--webserver_interface=127.23.72.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.189.keytab
--principal=oryx/127.23.72.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:35011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:36587,127.23.72.189:38137
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:57.110352 31000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:57.110702 31000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:57.110780 31000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:57.114820 31000 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:22:57.114876 31000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:57.114895 31000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:57.114939 31000 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:22:57.114972 31000 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:22:57.120386 31000 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:35011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:36587,127.23.72.189:38137
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.189.keytab
--principal=oryx/127.23.72.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.189:38137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.23.72.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31000
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
W20260808 18:22:57.120605 31000 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.23.72.190:36587,127.23.72.189:38137'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260808 18:22:57.121471 31000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:57.122494 31000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:57.130836 31007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:57.130949 31010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:57.130846 31008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:57.131114 31000 server_base.cc:1034] running on GCE node
I20260808 18:22:57.131770 31000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:57.132920 31000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:57.134112 31000 hybrid_clock.cc:648] HybridClock initialized: now 1786213377134089 us; error 44 us; skew 500 ppm
Aug 08 18:22:57 dist-test-slave-5k54 krb5kdc[30435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213377, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.72.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:57.137089 31000 init.cc:377] Logged in from keytab as oryx/127.23.72.189@KRBTEST.COM (short username oryx)
I20260808 18:22:57.137961 31000 webserver.cc:460] Webserver started at http://127.23.72.189:41585/ using document root <none> and password file <none>
I20260808 18:22:57.138514 31000 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:57.138604 31000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:57.138859 31000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:57.141028 31000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-1/data/instance:
uuid: "11d877a7b3c344f89673448718d22e9a"
format_stamp: "Formatted at 2026-08-08 18:22:57 on dist-test-slave-5k54"
server_key: "d16d1ef14d22ff81d6b121fc38d4ceca"
server_key_iv: "d211f53ab77102af0799276cd0f1d908"
server_key_version: "encryptionkey@0"
I20260808 18:22:57.141609 31000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-1/wal/instance:
uuid: "11d877a7b3c344f89673448718d22e9a"
format_stamp: "Formatted at 2026-08-08 18:22:57 on dist-test-slave-5k54"
server_key: "d16d1ef14d22ff81d6b121fc38d4ceca"
server_key_iv: "d211f53ab77102af0799276cd0f1d908"
server_key_version: "encryptionkey@0"
I20260808 18:22:57.145452 31000 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 18:22:57.147756 31018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:57.148959 31000 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 18:22:57.149098 31000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-1/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-1/wal
uuid: "11d877a7b3c344f89673448718d22e9a"
format_stamp: "Formatted at 2026-08-08 18:22:57 on dist-test-slave-5k54"
server_key: "d16d1ef14d22ff81d6b121fc38d4ceca"
server_key_iv: "d211f53ab77102af0799276cd0f1d908"
server_key_version: "encryptionkey@0"
I20260808 18:22:57.149214 31000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:57.170785 31000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:57.173830 31000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:57.174052 31000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:57.180958 31000 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.189:38137
I20260808 18:22:57.181236 31074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.189:38137 every 8 connection(s)
I20260808 18:22:57.181962 31000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-1/data/info.pb
I20260808 18:22:57.184774 31075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:57.186393 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 31000
I20260808 18:22:57.189090 31075 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 18:22:57 dist-test-slave-5k54 krb5kdc[30435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.23.72.189@KRBTEST.COM
Aug 08 18:22:57 dist-test-slave-5k54 krb5kdc[30435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213377, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.72.189@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM
I20260808 18:22:57.201586 30526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:57.191024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49358 (local address 127.23.72.190:36587)
0808 18:22:57.191150 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:57.191154 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:57.191507 (+ 353us) server_negotiation.cc:407] Connection header received
0808 18:22:57.191710 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:57.191713 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:57.191769 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:57.191861 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:57.192644 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.193443 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.194635 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.194864 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.194972 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:57.197999 (+ 3027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:57.198002 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:57.198004 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:57.198027 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:57.199773 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:57.200221 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:57.200225 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:57.200227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:57.200309 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:57.200582 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:57.200585 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:57.200587 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:57.200832 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:57.200926 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:57.201318 (+ 392us) server_negotiation.cc:299] Negotiation successful
0808 18:22:57.201438 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 18:22:57.201984 31080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:57.190968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:36587 (local address 127.0.0.1:49358)
0808 18:22:57.191368 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:57.191413 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:57.191590 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:57.191912 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:57.191918 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:57.192350 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:57.192499 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:57.192507 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.193578 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.193584 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:57.194503 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:57.194512 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.195005 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.195009 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:57.195117 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:57.196139 (+ 1022us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:57.196164 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:57.197848 (+ 1684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:57.199881 (+ 2033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:57.199888 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:57.199893 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:57.200120 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:57.200399 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:57.200402 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:57.200406 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:57.200492 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:57.200976 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:57.200982 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:57.201089 (+ 107us) client_negotiation.cc:769] Sending connection context
0808 18:22:57.201227 (+ 138us) client_negotiation.cc:240] Negotiation successful
0808 18:22:57.201380 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":109,"threads_started":1}
I20260808 18:22:57.201984 31079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:57.187954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40854 (local address 127.23.72.189:38137)
0808 18:22:57.188352 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:57.188368 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:57.188402 (+ 34us) server_negotiation.cc:407] Connection header received
0808 18:22:57.189007 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:57.189042 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:57.189358 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:57.189718 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:57.190533 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.192036 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.193047 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.193550 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.193705 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:57.196573 (+ 2868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:57.196579 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:57.196591 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:57.196615 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:57.198479 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:57.199284 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:57.199290 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:57.199296 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:57.199417 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:57.199733 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:57.199736 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:57.199738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:57.200171 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:57.200372 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:57.200758 (+ 386us) server_negotiation.cc:299] Negotiation successful
0808 18:22:57.201043 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":133,"threads_started":1}
I20260808 18:22:57.203344 31075 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.72.189" port: 38137 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 18:22:57 dist-test-slave-5k54 krb5kdc[30435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213377, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.72.189@KRBTEST.COM for oryx/127.23.72.189@KRBTEST.COM
I20260808 18:22:57.210922 31080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:57.203872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.189:38137 (local address 127.0.0.1:40860)
0808 18:22:57.204053 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:57.204066 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:57.204143 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:57.204388 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:57.204390 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:57.204514 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:57.204594 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:57.204598 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.205660 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.205662 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:57.206131 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:57.206137 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.206573 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.206576 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:57.206644 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:57.207090 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:57.207105 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:57.208277 (+ 1172us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:57.209679 (+ 1402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:57.209682 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:57.209683 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:57.209855 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:57.210079 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:57.210081 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:57.210082 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:57.210149 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:57.210446 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:57.210447 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:57.210512 (+ 65us) client_negotiation.cc:769] Sending connection context
0808 18:22:57.210586 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 18:22:57.210716 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124}
I20260808 18:22:57.210956 31079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:57.203948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40860 (local address 127.23.72.189:38137)
0808 18:22:57.204042 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:57.204046 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:57.204105 (+ 59us) server_negotiation.cc:407] Connection header received
0808 18:22:57.204238 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:57.204241 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:57.204286 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:57.204360 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:57.204702 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.205568 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.206250 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.206475 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.206559 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:57.208394 (+ 1835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:57.208396 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:57.208398 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:57.208418 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:57.209586 (+ 1168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:57.209935 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:57.209937 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:57.209938 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:57.210004 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:57.210220 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:57.210222 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:57.210223 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:57.210371 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:57.210440 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:57.210640 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 18:22:57.210808 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":28}
I20260808 18:22:57.213430 31075 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a: Bootstrap starting.
I20260808 18:22:57.215770 31075 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:57.216500 31075 log.cc:824] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:57.218431 31075 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a: No bootstrap required, opened a new log
I20260808 18:22:57.221310 31075 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } peers { permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 38137 } }
I20260808 18:22:57.221527 31075 raft_consensus.cc:374] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:57.221606 31075 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11d877a7b3c344f89673448718d22e9a, State: Initialized, Role: FOLLOWER
I20260808 18:22:57.222011 31075 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } peers { permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 38137 } }
I20260808 18:22:57.222486 31082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } peers { permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 38137 } } }
I20260808 18:22:57.222595 31082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:22:57.223116 31075 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [sys.catalog]: configured and running, proceeding with master startup.
W20260808 18:22:57.227720 31095 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:22:57.227808 31095 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:22:57.229597 31000 master_runner.cc:428] Detected that this master 11d877a7b3c344f89673448718d22e9a is joining an existing cluster
I20260808 18:22:57.229667 31000 master_runner.cc:432] Initiating AddMaster RPC to add 127.23.72.189:38137
I20260808 18:22:57.239815 31079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:57.231708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40868 (local address 127.23.72.189:38137)
0808 18:22:57.231832 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:57.231837 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:57.232119 (+ 282us) server_negotiation.cc:407] Connection header received
0808 18:22:57.232219 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:57.232222 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:57.232268 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:57.232353 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:57.232906 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.233681 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.234695 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.234994 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.235114 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:57.236351 (+ 1237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:57.236355 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:57.236357 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:57.236405 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:57.238143 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:57.238618 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:57.238623 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:57.238625 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:57.238703 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:57.238975 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:57.238981 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:57.238983 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:57.239144 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:57.239252 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:57.239528 (+ 276us) server_negotiation.cc:299] Negotiation successful
0808 18:22:57.239683 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":45,"spinlock_wait_cycles":33152}
I20260808 18:22:57.240075 30526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:57.231629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49374 (local address 127.23.72.190:36587)
0808 18:22:57.231763 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:57.231768 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:57.232100 (+ 332us) server_negotiation.cc:407] Connection header received
0808 18:22:57.232249 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:57.232252 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:57.232301 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:57.232378 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:57.232940 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.233466 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.234643 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.235023 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.235126 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:57.236552 (+ 1426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:57.236556 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:57.236560 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:57.236588 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:57.238254 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:57.238746 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:57.238751 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:57.238755 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:57.238839 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:57.239152 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:57.239156 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:57.239158 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:57.239366 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:57.239471 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:57.239807 (+ 336us) server_negotiation.cc:299] Negotiation successful
0808 18:22:57.239919 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 18:22:57.239827 31104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:57.231716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.189:38137 (local address 127.0.0.1:40868)
0808 18:22:57.232017 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:57.232033 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:57.232138 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:57.232382 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:57.232385 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:57.232629 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:57.232760 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:57.232770 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.233802 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.233805 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:57.234614 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:57.234623 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.235128 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.235131 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:57.235224 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:57.235731 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:57.235742 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:57.236216 (+ 474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:57.238262 (+ 2046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:57.238264 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:57.238266 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:57.238525 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:57.238797 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:57.238802 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:57.238806 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:57.238903 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:57.239240 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:57.239243 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:57.239345 (+ 102us) client_negotiation.cc:769] Sending connection context
0808 18:22:57.239490 (+ 145us) client_negotiation.cc:240] Negotiation successful
0808 18:22:57.239630 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":102,"threads_started":1}
I20260808 18:22:57.240533 31105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:57.231600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:36587 (local address 127.0.0.1:49374)
0808 18:22:57.232017 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:57.232033 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:57.232149 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:57.232531 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:57.232533 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:57.232705 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:57.232807 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:57.232816 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.233576 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.233581 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:57.234515 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:57.234525 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.235245 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.235249 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:57.235344 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:57.235925 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:57.235940 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:57.236407 (+ 467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:57.238362 (+ 1955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:57.238365 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:57.238368 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:57.238659 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:57.238937 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:57.238940 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:57.238942 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:57.239034 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:57.239546 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:57.239549 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:57.239633 (+ 84us) client_negotiation.cc:769] Sending connection context
0808 18:22:57.240207 (+ 574us) client_negotiation.cc:240] Negotiation successful
0808 18:22:57.240342 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":126,"threads_started":1}
Aug 08 18:22:57 dist-test-slave-5k54 krb5kdc[30435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213376, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.72.190@KRBTEST.COM for oryx/127.23.72.189@KRBTEST.COM
I20260808 18:22:57.253238 31079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:57.244722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40880 (local address 127.23.72.189:38137)
0808 18:22:57.244863 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:57.244869 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:57.245197 (+ 328us) server_negotiation.cc:407] Connection header received
0808 18:22:57.245421 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:57.245424 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:57.245471 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:57.245564 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:57.246220 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.246686 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.247596 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.247820 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.247933 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:57.249973 (+ 2040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:57.249977 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:57.249980 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:57.250002 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:57.251334 (+ 1332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:57.251827 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:57.251832 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:57.251834 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:57.251936 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:57.252300 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:57.252303 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:57.252305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:57.252504 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:57.252579 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:57.252953 (+ 374us) server_negotiation.cc:299] Negotiation successful
0808 18:22:57.253070 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 18:22:57.253275 31106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:57.244698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.189:38137 (local address 127.0.0.1:40880)
0808 18:22:57.245082 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:57.245121 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:57.245265 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:57.245599 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:57.245605 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:57.246011 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:57.246112 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:57.246117 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.246805 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.246810 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:57.247477 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:57.247487 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.247935 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.247939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:57.248057 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:57.248519 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:57.248530 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:57.249824 (+ 1294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:57.251488 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:57.251497 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:57.251503 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:57.251726 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:57.252056 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:57.252059 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:57.252061 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:57.252162 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:57.252606 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:57.252612 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:57.252739 (+ 127us) client_negotiation.cc:769] Sending connection context
0808 18:22:57.252893 (+ 154us) client_negotiation.cc:240] Negotiation successful
0808 18:22:57.253103 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":107,"threads_started":1}
I20260808 18:22:57.254642 30484 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.23.72.189:38137: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "11d877a7b3c344f89673448718d22e9a"
member_type: NON_VOTER
last_known_addr {
host: "127.23.72.189"
port: 38137
}
attrs {
promote: true
}
}
dest_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18"
cas_config_opid_index: -1
I20260808 18:22:57.255362 30484 consensus_queue.cc:237] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } peers { permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: NON_VOTER last_known_addr { host: "127.23.72.189" port: 38137 } attrs { promote: true } }
I20260808 18:22:57.257022 31107 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 11d877a7b3c344f89673448718d22e9a (127.23.72.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } peers { permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: NON_VOTER last_known_addr { host: "127.23.72.189" port: 38137 } attrs { promote: true } } }
I20260808 18:22:57.258090 31048 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:57.259264 31107 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.23.72.189:38137
I20260808 18:22:57.259506 31048 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [term 1 FOLLOWER]: Refusing update from remote peer e9d826ba0df445bfb8cc41e8dc8c9f18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260808 18:22:57.259697 31000 master.cc:537] Master@127.23.72.189:38137 shutting down...
I20260808 18:22:57.260073 31108 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } peers { permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: NON_VOTER last_known_addr { host: "127.23.72.189" port: 38137 } attrs { promote: true } } }
I20260808 18:22:57.260193 31108 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:57.260125 31107 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: NON_VOTER last_known_addr { host: "127.23.72.189" port: 38137 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 18:22:57.262518 30469 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260808 18:22:57.263387 30469 consensus_peers.cc:487] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 -> Peer 11d877a7b3c344f89673448718d22e9a (127.23.72.189:38137): Couldn't send request to peer 11d877a7b3c344f89673448718d22e9a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 18:22:57.264160 31000 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:22:57.264434 31000 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:22:57.264550 31000 tablet_replica.cc:330] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a: stopping tablet replica
I20260808 18:22:57.268718 31000 master.cc:552] Master@127.23.72.189:38137 shutdown complete.
I20260808 18:22:57.268770 31000 master_runner.cc:305] Clearing existing system tablet
I20260808 18:22:57.269004 31000 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:57.269061 31000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:57.270358 31000 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 18:22:57.271302 31113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:57.271569 31000 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:57.271829 31000 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 18:22:57.271914 31000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-1/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-1/wal
uuid: "11d877a7b3c344f89673448718d22e9a"
format_stamp: "Formatted at 2026-08-08 18:22:57 on dist-test-slave-5k54"
server_key: "d16d1ef14d22ff81d6b121fc38d4ceca"
server_key_iv: "d211f53ab77102af0799276cd0f1d908"
server_key_version: "encryptionkey@0"
WARNING: no policy specified for oryx/127.23.72.188@KRBTEST.COM; defaulting to no policy
I20260808 18:22:57.274124 31000 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a: Deleting tablet data with delete state TABLET_DATA_DELETED
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.72.188@KRBTEST.COM" created.
I20260808 18:22:57.276506 31000 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 18:22:57.276597 31000 log.cc:1196] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260808 18:22:57.276844 31000 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a: Deleting consensus metadata
I20260808 18:22:57.277009 31000 master_runner.cc:315] Copying system tablet from 127.23.72.190:36587
I20260808 18:22:57.278131 31000 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a: tablet copy: Beginning tablet copy session from remote peer at address 127.23.72.190:36587
I20260808 18:22:57.287890 30526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:57.278799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49376 (local address 127.23.72.190:36587)
0808 18:22:57.278924 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:57.278929 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:57.279167 (+ 238us) server_negotiation.cc:407] Connection header received
0808 18:22:57.279287 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:57.279290 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:57.279342 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:57.279467 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:57.280035 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.280857 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.282211 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.282557 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.282660 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:57.284110 (+ 1450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:57.284114 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:57.284117 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:57.284144 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:57.285983 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:57.286581 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:57.286584 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:57.286586 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:57.286692 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:57.286987 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:57.286993 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:57.286997 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:57.287187 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:57.287269 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:57.287653 (+ 384us) server_negotiation.cc:299] Negotiation successful
0808 18:22:57.287753 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 18:22:57.288305 31121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:57.278700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:36587 (local address 127.0.0.1:49376)
0808 18:22:57.279054 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:57.279073 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:57.279198 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:57.279504 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:57.279510 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:57.279746 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:57.279902 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:57.279913 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.280981 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.280985 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:57.282071 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:57.282086 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.282686 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.282692 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:57.282806 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:57.283405 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:57.283476 (+ 71us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:57.283946 (+ 470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:57.286106 (+ 2160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:57.286109 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:57.286112 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:57.286435 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:57.286800 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:57.286803 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:57.286805 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:57.286896 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:57.287281 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:57.287284 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:57.287379 (+ 95us) client_negotiation.cc:769] Sending connection context
0808 18:22:57.287550 (+ 171us) client_negotiation.cc:240] Negotiation successful
0808 18:22:57.287688 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":88,"threads_started":1}
I20260808 18:22:57.288892 30507 tablet_copy_service.cc:140] P e9d826ba0df445bfb8cc41e8dc8c9f18: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 11d877a7b3c344f89673448718d22e9a ({username='oryx', principal='oryx/127.23.72.189@KRBTEST.COM'} at 127.0.0.1:49376)
I20260808 18:22:57.289064 30507 tablet_copy_service.cc:161] P e9d826ba0df445bfb8cc41e8dc8c9f18: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 11d877a7b3c344f89673448718d22e9a at {username='oryx', principal='oryx/127.23.72.189@KRBTEST.COM'} at 127.0.0.1:49376: session id = 11d877a7b3c344f89673448718d22e9a-00000000000000000000000000000000
I20260808 18:22:57.290712 30507 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 18:22:57.292285 31000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.72.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.188.keytab.
Entry for principal oryx/127.23.72.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.188.keytab.
I20260808 18:22:57.294945 31000 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a: tablet copy: Starting download of 0 data blocks...
I20260808 18:22:57.295135 31000 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a: tablet copy: Starting download of 1 WAL segments...
I20260808 18:22:57.295125 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.188:37769
--webserver_interface=127.23.72.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.188.keytab
--principal=oryx/127.23.72.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:35011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:36587,127.23.72.189:38137,127.23.72.188:37769
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:22:57.296819 31000 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 18:22:57.298480 30507 tablet_copy_service.cc:342] P e9d826ba0df445bfb8cc41e8dc8c9f18: Request end of tablet copy session 11d877a7b3c344f89673448718d22e9a-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.23.72.189@KRBTEST.COM'} at 127.0.0.1:49376
I20260808 18:22:57.298596 30507 tablet_copy_service.cc:433] P e9d826ba0df445bfb8cc41e8dc8c9f18: ending tablet copy session 11d877a7b3c344f89673448718d22e9a-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 11d877a7b3c344f89673448718d22e9a
I20260808 18:22:57.306370 31000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:57.308029 31123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:57.308264 31124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:57.311489 31126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:57.316303 31000 server_base.cc:1034] running on GCE node
I20260808 18:22:57.316572 31000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:57.316860 31000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:57.317997 31000 hybrid_clock.cc:648] HybridClock initialized: now 1786213377317980 us; error 32 us; skew 500 ppm
Aug 08 18:22:57 dist-test-slave-5k54 krb5kdc[30435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213377, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.72.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:57.319638 31000 init.cc:377] Logged in from keytab as oryx/127.23.72.189@KRBTEST.COM (short username oryx)
I20260808 18:22:57.320096 31000 webserver.cc:460] Webserver started at http://127.23.72.189:38185/ using document root <none> and password file <none>
I20260808 18:22:57.320279 31000 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:57.320323 31000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:57.321609 31000 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 18:22:57.322625 31134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:57.322898 31000 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 18:22:57.322976 31000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-1/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-1/wal
uuid: "11d877a7b3c344f89673448718d22e9a"
format_stamp: "Formatted at 2026-08-08 18:22:57 on dist-test-slave-5k54"
server_key: "d16d1ef14d22ff81d6b121fc38d4ceca"
server_key_iv: "d211f53ab77102af0799276cd0f1d908"
server_key_version: "encryptionkey@0"
I20260808 18:22:57.323060 31000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:57.335490 31000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:57.337706 31000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:57.342438 31000 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.189:38137
I20260808 18:22:57.342542 31191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.189:38137 every 8 connection(s)
I20260808 18:22:57.343027 31000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-1/data/info.pb
I20260808 18:22:57.345184 31192 sys_catalog.cc:263] Verifying existing consensus state
I20260808 18:22:57.346191 31192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a: Bootstrap starting.
I20260808 18:22:57.356493 31192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 18:22:57.356819 31192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a: Bootstrap complete.
I20260808 18:22:57.357483 31192 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } peers { permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: NON_VOTER last_known_addr { host: "127.23.72.189" port: 38137 } attrs { promote: true } }
I20260808 18:22:57.357622 31192 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 11d877a7b3c344f89673448718d22e9a, State: Initialized, Role: LEARNER
I20260808 18:22:57.357759 31192 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } peers { permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: NON_VOTER last_known_addr { host: "127.23.72.189" port: 38137 } attrs { promote: true } }
I20260808 18:22:57.358158 31196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } peers { permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: NON_VOTER last_known_addr { host: "127.23.72.189" port: 38137 } attrs { promote: true } } }
I20260808 18:22:57.358275 31192 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:22:57.358364 31196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [sys.catalog]: This master's current role is: LEARNER
I20260808 18:22:57.364663 31207 catalog_manager.cc:1259] Loaded cluster ID: bf3f1e45a3cf47969ef2d1762c680e0e
I20260808 18:22:57.364727 31207 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a: loading cluster ID for follower catalog manager: success
I20260808 18:22:57.366735 31207 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a: acquiring CA information for follower catalog manager: success
I20260808 18:22:57.368073 31207 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 18:22:57.420342 31122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:57.420590 31122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:57.420629 31122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:57.424700 31122 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:22:57.424770 31122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:57.424809 31122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:57.424853 31122 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:22:57.424888 31122 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:22:57.430197 31122 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:35011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:36587,127.23.72.189:38137,127.23.72.188:37769
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.188.keytab
--principal=oryx/127.23.72.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.188:37769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.23.72.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:57.431249 31122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:57.432303 31122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:57.440907 31214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:57.440929 31213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:57.440904 31216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:57.440939 31122 server_base.cc:1034] running on GCE node
I20260808 18:22:57.441689 31122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:57.442723 31122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:57.443907 31122 hybrid_clock.cc:648] HybridClock initialized: now 1786213377443890 us; error 29 us; skew 500 ppm
Aug 08 18:22:57 dist-test-slave-5k54 krb5kdc[30435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213377, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.72.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:57.446844 31122 init.cc:377] Logged in from keytab as oryx/127.23.72.188@KRBTEST.COM (short username oryx)
I20260808 18:22:57.447708 31122 webserver.cc:460] Webserver started at http://127.23.72.188:45713/ using document root <none> and password file <none>
I20260808 18:22:57.448246 31122 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:57.448336 31122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:57.448565 31122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:57.450517 31122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-2/data/instance:
uuid: "9d31cd09bc7843f4854913a5b460ba84"
format_stamp: "Formatted at 2026-08-08 18:22:57 on dist-test-slave-5k54"
server_key: "b8147d5581a5369cb479599d0a7df80e"
server_key_iv: "689255d02cc0d49d02de848000caa3a2"
server_key_version: "encryptionkey@0"
I20260808 18:22:57.451082 31122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-2/wal/instance:
uuid: "9d31cd09bc7843f4854913a5b460ba84"
format_stamp: "Formatted at 2026-08-08 18:22:57 on dist-test-slave-5k54"
server_key: "b8147d5581a5369cb479599d0a7df80e"
server_key_iv: "689255d02cc0d49d02de848000caa3a2"
server_key_version: "encryptionkey@0"
I20260808 18:22:57.454763 31122 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 18:22:57.457111 31223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:57.458791 31122 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260808 18:22:57.458950 31122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-2/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-2/wal
uuid: "9d31cd09bc7843f4854913a5b460ba84"
format_stamp: "Formatted at 2026-08-08 18:22:57 on dist-test-slave-5k54"
server_key: "b8147d5581a5369cb479599d0a7df80e"
server_key_iv: "689255d02cc0d49d02de848000caa3a2"
server_key_version: "encryptionkey@0"
I20260808 18:22:57.459056 31122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:57.480263 31122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:57.483355 31122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:57.483610 31122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:57.491323 31122 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.188:37769
I20260808 18:22:57.491662 31283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.188:37769 every 8 connection(s)
I20260808 18:22:57.492516 31122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-2/data/info.pb
I20260808 18:22:57.495234 31284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:57.498036 30694 heartbeater.cc:503] Master 127.23.72.190:36587 was elected leader, sending a full tablet report...
I20260808 18:22:57.498881 31284 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:22:57.502414 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 31122
Aug 08 18:22:57 dist-test-slave-5k54 krb5kdc[30435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213377, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.72.188@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM
Aug 08 18:22:57 dist-test-slave-5k54 krb5kdc[30435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.23.72.188@KRBTEST.COM
I20260808 18:22:57.512885 30526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:57.501064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49386 (local address 127.23.72.190:36587)
0808 18:22:57.501217 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:57.501221 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:57.501965 (+ 744us) server_negotiation.cc:407] Connection header received
0808 18:22:57.502949 (+ 984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:57.502952 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:57.503008 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:57.503122 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:57.504410 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.504952 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.506162 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.506396 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.506500 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:57.508996 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:57.508999 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:57.509002 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:57.509031 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:57.510548 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:57.511094 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:57.511096 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:57.511098 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:57.511169 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:57.511557 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:57.511563 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:57.511568 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:57.511796 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:57.511935 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:57.512523 (+ 588us) server_negotiation.cc:299] Negotiation successful
0808 18:22:57.512662 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 18:22:57.513768 31287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:57.503294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44998 (local address 127.23.72.188:37769)
0808 18:22:57.503630 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:57.503639 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:57.503658 (+ 19us) server_negotiation.cc:407] Connection header received
0808 18:22:57.503692 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:57.503696 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:57.503871 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:57.504030 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:57.504783 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.505662 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.506491 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.506833 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.506946 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:57.509381 (+ 2435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:57.509389 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:57.509405 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:57.509436 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:57.511387 (+ 1951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:57.511868 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:57.511873 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:57.511892 (+ 19us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:57.511979 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:57.512232 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:57.512235 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:57.512237 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:57.512503 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:57.512686 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:57.512897 (+ 211us) server_negotiation.cc:299] Negotiation successful
0808 18:22:57.513021 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":237,"spinlock_wait_cycles":2304,"thread_start_us":135,"threads_started":1}
I20260808 18:22:57.513768 31286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:57.501378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:36587 (local address 127.0.0.1:49386)
0808 18:22:57.501806 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:57.501865 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:57.502745 (+ 880us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:57.503303 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:57.503312 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:57.503760 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:57.504264 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:57.504276 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.505089 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.505093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:57.506038 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:57.506048 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.506543 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.506547 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:57.506709 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:57.507321 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:57.507343 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:57.508856 (+ 1513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:57.510654 (+ 1798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:57.510661 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:57.510670 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:57.511003 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:57.511259 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:57.511264 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:57.511267 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:57.511475 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:57.511925 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:57.511934 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:57.512235 (+ 301us) client_negotiation.cc:769] Sending connection context
0808 18:22:57.512453 (+ 218us) client_negotiation.cc:240] Negotiation successful
0808 18:22:57.512792 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"mutex_wait_us":173,"spinlock_wait_cycles":4480,"thread_start_us":111,"threads_started":1}
I20260808 18:22:57.515357 31284 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.72.189" port: 38137 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 18:22:57 dist-test-slave-5k54 krb5kdc[30435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213377, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.72.188@KRBTEST.COM for oryx/127.23.72.189@KRBTEST.COM
I20260808 18:22:57.523581 31288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:57.516048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40886 (local address 127.23.72.189:38137)
0808 18:22:57.516337 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:57.516340 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:57.516353 (+ 13us) server_negotiation.cc:407] Connection header received
0808 18:22:57.516388 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:57.516390 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:57.516429 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:57.516503 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:57.517016 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.517672 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.518369 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.518593 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.518667 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:57.520569 (+ 1902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:57.520572 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:57.520574 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:57.520597 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:57.522061 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:57.522469 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:57.522471 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:57.522472 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:57.522544 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:57.522812 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:57.522814 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:57.522815 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:57.522974 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:57.523046 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:57.523266 (+ 220us) server_negotiation.cc:299] Negotiation successful
0808 18:22:57.523373 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":114,"threads_started":1}
I20260808 18:22:57.523584 31286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:57.515934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.189:38137 (local address 127.0.0.1:40886)
0808 18:22:57.516085 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:57.516099 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:57.516223 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:57.516574 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:57.516580 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:57.516780 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:57.516881 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:57.516886 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.517767 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.517770 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:57.518272 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:57.518279 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.518682 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.518684 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:57.518753 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:57.519129 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:57.519140 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:57.520441 (+ 1301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:57.522172 (+ 1731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:57.522174 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:57.522176 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:57.522383 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:57.522625 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:57.522627 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:57.522628 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:57.522706 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:57.523059 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:57.523061 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:57.523140 (+ 79us) client_negotiation.cc:769] Sending connection context
0808 18:22:57.523232 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 18:22:57.523401 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92}
I20260808 18:22:57.524375 31284 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.72.188" port: 37769 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 18:22:57 dist-test-slave-5k54 krb5kdc[30435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213377, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.72.188@KRBTEST.COM for oryx/127.23.72.188@KRBTEST.COM
I20260808 18:22:57.531687 31286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:57.524897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:37769 (local address 127.0.0.1:45012)
0808 18:22:57.525018 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:57.525033 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:57.525102 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:57.525284 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:57.525286 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:57.525412 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:57.525476 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:57.525480 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.526178 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.526180 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:57.526650 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:57.526657 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.527116 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.527118 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:57.527171 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:57.527602 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:57.527612 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:57.528712 (+ 1100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:57.530332 (+ 1620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:57.530334 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:57.530335 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:57.530502 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:57.530764 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:57.530766 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:57.530767 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:57.530913 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:57.531230 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:57.531233 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:57.531308 (+ 75us) client_negotiation.cc:769] Sending connection context
0808 18:22:57.531391 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 18:22:57.531562 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 18:22:57.531728 31287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:57.524956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45012 (local address 127.23.72.188:37769)
0808 18:22:57.525078 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:57.525082 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:57.525104 (+ 22us) server_negotiation.cc:407] Connection header received
0808 18:22:57.525160 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:57.525162 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:57.525195 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:57.525261 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:57.525569 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.526091 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.526744 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.527022 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.527104 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:57.528833 (+ 1729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:57.528835 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:57.528837 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:57.528857 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:57.530234 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:57.530582 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:57.530584 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:57.530585 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:57.530655 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:57.530992 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:57.530994 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:57.530995 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:57.531142 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:57.531211 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:57.531497 (+ 286us) server_negotiation.cc:299] Negotiation successful
0808 18:22:57.531614 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":52}
I20260808 18:22:57.534792 31284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84: Bootstrap starting.
I20260808 18:22:57.537178 31284 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:57.537899 31284 log.cc:824] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:57.539846 31284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84: No bootstrap required, opened a new log
I20260808 18:22:57.542697 31284 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } peers { permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 38137 } } peers { permanent_uuid: "9d31cd09bc7843f4854913a5b460ba84" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 37769 } }
I20260808 18:22:57.542922 31284 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:57.543007 31284 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d31cd09bc7843f4854913a5b460ba84, State: Initialized, Role: FOLLOWER
I20260808 18:22:57.543514 31284 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } peers { permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 38137 } } peers { permanent_uuid: "9d31cd09bc7843f4854913a5b460ba84" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 37769 } }
I20260808 18:22:57.544018 31290 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } peers { permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 38137 } } peers { permanent_uuid: "9d31cd09bc7843f4854913a5b460ba84" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 37769 } } }
I20260808 18:22:57.544144 31290 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:22:57.544837 31284 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 18:22:57.549230 31304 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:22:57.549324 31304 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:22:57.552608 31122 master_runner.cc:428] Detected that this master 9d31cd09bc7843f4854913a5b460ba84 is joining an existing cluster
I20260808 18:22:57.552683 31122 master_runner.cc:432] Initiating AddMaster RPC to add 127.23.72.188:37769
I20260808 18:22:57.564013 30526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:57.555038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49390 (local address 127.23.72.190:36587)
0808 18:22:57.555186 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:57.555191 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:57.555206 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:22:57.555247 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:57.555250 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:57.555299 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:57.555384 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:57.556043 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.556895 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.558182 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.558554 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.558638 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:57.560185 (+ 1547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:57.560189 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:57.560192 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:57.560217 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:57.561978 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:57.562536 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:57.562540 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:57.562542 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:57.562630 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:57.562926 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:57.562929 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:57.562931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:57.563136 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:57.563225 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:57.563731 (+ 506us) server_negotiation.cc:299] Negotiation successful
0808 18:22:57.563853 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 18:22:57.564375 31287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:57.555208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45020 (local address 127.23.72.188:37769)
0808 18:22:57.555292 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:57.555298 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:57.555340 (+ 42us) server_negotiation.cc:407] Connection header received
0808 18:22:57.555512 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:57.555515 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:57.555558 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:57.555605 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:57.556192 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.556891 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.558563 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.559149 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.559263 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:57.560793 (+ 1530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:57.560798 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:57.560801 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:57.560825 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:57.562604 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:57.563081 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:57.563085 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:57.563087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:57.563164 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:57.563460 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:57.563463 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:57.563465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:57.563740 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:57.563829 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:57.564074 (+ 245us) server_negotiation.cc:299] Negotiation successful
0808 18:22:57.564189 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":267,"server-negotiator.queue_time_us":35}
I20260808 18:22:57.564556 31311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:57.554662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:36587 (local address 127.0.0.1:49390)
0808 18:22:57.554972 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:57.554987 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:57.555086 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:57.555551 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:57.555556 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:57.555797 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:57.555931 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:57.555939 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.557014 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.557017 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:57.558035 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:57.558046 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.558783 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.558787 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:57.558886 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:57.559504 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:57.559523 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:57.560042 (+ 519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:57.562094 (+ 2052us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:57.562098 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:57.562100 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:57.562407 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:57.562708 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:57.562711 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:57.562713 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:57.562830 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:57.563273 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:57.563276 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:57.563375 (+ 99us) client_negotiation.cc:769] Sending connection context
0808 18:22:57.563623 (+ 248us) client_negotiation.cc:240] Negotiation successful
0808 18:22:57.564258 (+ 635us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":493,"spinlock_wait_cycles":768,"thread_start_us":119,"threads_started":1}
I20260808 18:22:57.564857 31310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:57.554696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:37769 (local address 127.0.0.1:45020)
0808 18:22:57.555270 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:57.555282 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:57.555394 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:57.555727 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:57.555729 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:57.555965 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:57.556087 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:57.556094 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.557014 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.557017 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:57.558409 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:57.558467 (+ 58us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.559578 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.559584 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:57.559744 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:57.560265 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:57.560277 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:57.560688 (+ 411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:57.562716 (+ 2028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:57.562719 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:57.562720 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:57.562985 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:57.563246 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:57.563252 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:57.563254 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:57.563345 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:57.563820 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:57.563823 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:57.563904 (+ 81us) client_negotiation.cc:769] Sending connection context
0808 18:22:57.564018 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 18:22:57.564681 (+ 663us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"mutex_wait_us":25,"thread_start_us":96,"threads_started":1}
I20260808 18:22:57.567219 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 30551
Aug 08 18:22:57 dist-test-slave-5k54 krb5kdc[30435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213376, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.72.190@KRBTEST.COM for oryx/127.23.72.188@KRBTEST.COM
I20260808 18:22:57.578330 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.129:38285
--local_ip_for_outbound_sockets=127.23.72.129
--tserver_master_addrs=127.23.72.190:36587
--webserver_port=43901
--webserver_interface=127.23.72.129
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.129.keytab
--principal=oryx/127.23.72.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:35011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.23.72.190:36587,127.23.72.189:38137
--tserver_master_addrs=127.23.72.190:36587,127.23.72.189:38137,127.23.72.188:37769 with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:22:57.591336 31106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:57.568913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:37769 (local address 127.0.0.1:45030)
0808 18:22:57.571942 (+ 3029us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:57.571958 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:57.572035 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:57.572900 (+ 865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:57.572904 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:57.573110 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:57.573243 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:57.573253 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.574431 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.574435 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:57.575322 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:57.575333 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.576022 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.576026 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:57.576125 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:57.576641 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:57.576659 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:57.578227 (+ 1568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:57.589543 (+ 11316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:57.589547 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:57.589549 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:57.589821 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:57.590173 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:57.590178 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:57.590180 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:57.590302 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:57.590771 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:57.590785 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:57.590890 (+ 105us) client_negotiation.cc:769] Sending connection context
0808 18:22:57.590972 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 18:22:57.591136 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2943}
I20260808 18:22:57.592010 31287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:57.571802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45030 (local address 127.23.72.188:37769)
0808 18:22:57.572609 (+ 807us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:57.572615 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:57.572631 (+ 16us) server_negotiation.cc:407] Connection header received
0808 18:22:57.572699 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:57.572704 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:57.572767 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:57.573357 (+ 590us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:57.573483 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.574299 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.575481 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.575896 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.576017 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:57.578380 (+ 2363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:57.578384 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:57.578386 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:57.578412 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:57.589389 (+ 10977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:57.589942 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:57.589946 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:57.589948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:57.590024 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:57.590415 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:57.590418 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:57.590420 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:57.590651 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:57.591665 (+ 1014us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:57.591740 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 18:22:57.591863 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":718}
I20260808 18:22:57.592585 31288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:57.555551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40892 (local address 127.23.72.189:38137)
0808 18:22:57.555667 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:57.555674 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:57.556262 (+ 588us) server_negotiation.cc:407] Connection header received
0808 18:22:57.556425 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:57.556428 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:57.556473 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:57.556522 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:57.557157 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.557915 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.559113 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.559492 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.559587 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:57.588674 (+ 29087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:57.588680 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:57.588686 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:57.588727 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:57.590424 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:57.591039 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:57.591045 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:57.591048 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:57.591146 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:57.591546 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:57.591552 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:57.591555 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:57.591783 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:57.591865 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:57.592287 (+ 422us) server_negotiation.cc:299] Negotiation successful
0808 18:22:57.592433 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 18:22:57.592942 30484 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.23.72.188:37769: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "9d31cd09bc7843f4854913a5b460ba84"
member_type: NON_VOTER
last_known_addr {
host: "127.23.72.188"
port: 37769
}
attrs {
promote: true
}
}
dest_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18"
cas_config_opid_index: 5
I20260808 18:22:57.593072 31312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:57.555776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.189:38137 (local address 127.0.0.1:40892)
0808 18:22:57.556184 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:57.556196 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:57.556322 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:57.556651 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:57.556654 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:57.556900 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:57.557022 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:57.557029 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.558053 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.558056 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:57.558971 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:57.558982 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.559769 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.559772 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:57.559874 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:57.587936 (+ 28062us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:57.587953 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:57.588483 (+ 530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:57.590557 (+ 2074us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:57.590562 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:57.590567 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:57.590889 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:57.591277 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:57.591282 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:57.591287 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:57.591391 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:57.591962 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:57.591968 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:57.592092 (+ 124us) client_negotiation.cc:769] Sending connection context
0808 18:22:57.592731 (+ 639us) client_negotiation.cc:240] Negotiation successful
0808 18:22:57.592885 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":97,"threads_started":1}
I20260808 18:22:57.593580 30484 consensus_queue.cc:237] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } peers { permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: NON_VOTER last_known_addr { host: "127.23.72.189" port: 38137 } attrs { promote: true } } peers { permanent_uuid: "9d31cd09bc7843f4854913a5b460ba84" member_type: NON_VOTER last_known_addr { host: "127.23.72.188" port: 37769 } attrs { promote: true } }
I20260808 18:22:57.594763 31107 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 9d31cd09bc7843f4854913a5b460ba84 (127.23.72.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } peers { permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: NON_VOTER last_known_addr { host: "127.23.72.189" port: 38137 } attrs { promote: true } } peers { permanent_uuid: "9d31cd09bc7843f4854913a5b460ba84" member_type: NON_VOTER last_known_addr { host: "127.23.72.188" port: 37769 } attrs { promote: true } } }
I20260808 18:22:57.596452 31107 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.23.72.188:37769
I20260808 18:22:57.596643 31107 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } peers { permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: NON_VOTER last_known_addr { host: "127.23.72.189" port: 38137 } attrs { promote: true } } peers { permanent_uuid: "9d31cd09bc7843f4854913a5b460ba84" member_type: NON_VOTER last_known_addr { host: "127.23.72.188" port: 37769 } attrs { promote: true } } }
I20260808 18:22:57.596755 31107 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:57.597118 31122 master.cc:537] Master@127.23.72.188:37769 shutting down...
I20260808 18:22:57.597730 31255 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:57.599218 31255 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [term 1 FOLLOWER]: Refusing update from remote peer e9d826ba0df445bfb8cc41e8dc8c9f18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 18:22:57.599740 31106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:57.594749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.189:38137 (local address 127.0.0.1:40894)
0808 18:22:57.594870 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:57.594885 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:57.594966 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:57.595361 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:57.595364 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:57.595383 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:22:57.595583 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:57.595590 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.596565 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.596570 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:57.597881 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:57.598022 (+ 141us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.599228 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.599235 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:57.599334 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:57.599381 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 18:22:57.599522 (+ 141us) client_negotiation.cc:240] Negotiation successful
0808 18:22:57.599594 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 18:22:57.599897 31108 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d31cd09bc7843f4854913a5b460ba84" member_type: NON_VOTER last_known_addr { host: "127.23.72.188" port: 37769 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:22:57.600355 31288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:57.594976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40894 (local address 127.23.72.189:38137)
0808 18:22:57.595093 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:57.595098 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:57.595112 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:22:57.595155 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:57.595160 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:57.595222 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:57.595332 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:22:57.595700 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.596418 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.598157 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.598759 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.598895 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:57.599059 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:57.600162 (+ 1103us) server_negotiation.cc:299] Negotiation successful
0808 18:22:57.600237 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 18:22:57.600767 31162 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [term 1 LEARNER]: Refusing update from remote peer e9d826ba0df445bfb8cc41e8dc8c9f18: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 18:22:57.601191 31108 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: NON_VOTER last_known_addr { host: "127.23.72.189" port: 38137 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260808 18:22:57.601692 30466 consensus_peers.cc:487] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 -> Peer 9d31cd09bc7843f4854913a5b460ba84 (127.23.72.188:37769): Couldn't send request to peer 9d31cd09bc7843f4854913a5b460ba84. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 18:22:57.602856 31122 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:22:57.603124 31122 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:22:57.603215 31122 tablet_replica.cc:330] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84: stopping tablet replica
I20260808 18:22:57.603264 31162 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 9d31cd09bc7843f4854913a5b460ba84 (127.23.72.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } peers { permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: NON_VOTER last_known_addr { host: "127.23.72.189" port: 38137 } attrs { promote: true } } peers { permanent_uuid: "9d31cd09bc7843f4854913a5b460ba84" member_type: NON_VOTER last_known_addr { host: "127.23.72.188" port: 37769 } attrs { promote: true } } }
I20260808 18:22:57.605659 31196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9d826ba0df445bfb8cc41e8dc8c9f18. Latest consensus state: current_term: 1 leader_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } peers { permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: NON_VOTER last_known_addr { host: "127.23.72.189" port: 38137 } attrs { promote: true } } peers { permanent_uuid: "9d31cd09bc7843f4854913a5b460ba84" member_type: NON_VOTER last_known_addr { host: "127.23.72.188" port: 37769 } attrs { promote: true } } }
I20260808 18:22:57.605784 31196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [sys.catalog]: This master's current role is: LEARNER
I20260808 18:22:57.605890 31196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } peers { permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: NON_VOTER last_known_addr { host: "127.23.72.189" port: 38137 } attrs { promote: true } } peers { permanent_uuid: "9d31cd09bc7843f4854913a5b460ba84" member_type: NON_VOTER last_known_addr { host: "127.23.72.188" port: 37769 } attrs { promote: true } } }
I20260808 18:22:57.606004 31196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [sys.catalog]: This master's current role is: LEARNER
I20260808 18:22:57.608105 31122 master.cc:552] Master@127.23.72.188:37769 shutdown complete.
I20260808 18:22:57.608193 31122 master_runner.cc:305] Clearing existing system tablet
I20260808 18:22:57.608474 31122 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:57.608573 31122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:57.610016 31122 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 18:22:57.610948 31317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:57.611140 31122 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:57.611392 31122 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 18:22:57.611532 31122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-2/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-2/wal
uuid: "9d31cd09bc7843f4854913a5b460ba84"
format_stamp: "Formatted at 2026-08-08 18:22:57 on dist-test-slave-5k54"
server_key: "b8147d5581a5369cb479599d0a7df80e"
server_key_iv: "689255d02cc0d49d02de848000caa3a2"
server_key_version: "encryptionkey@0"
I20260808 18:22:57.613533 31122 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:22:57.616160 31122 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 18:22:57.616310 31122 log.cc:1196] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260808 18:22:57.616612 31122 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84: Deleting consensus metadata
I20260808 18:22:57.616816 31122 master_runner.cc:315] Copying system tablet from 127.23.72.190:36587
I20260808 18:22:57.616819 31108 raft_consensus.cc:1050] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18: attempting to promote NON_VOTER 11d877a7b3c344f89673448718d22e9a to VOTER
I20260808 18:22:57.617312 31108 consensus_queue.cc:237] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } peers { permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 38137 } attrs { promote: false } } peers { permanent_uuid: "9d31cd09bc7843f4854913a5b460ba84" member_type: NON_VOTER last_known_addr { host: "127.23.72.188" port: 37769 } attrs { promote: true } }
I20260808 18:22:57.618062 31122 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84: tablet copy: Beginning tablet copy session from remote peer at address 127.23.72.190:36587
I20260808 18:22:57.618999 31106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:57.618612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:37769 (local address 127.0.0.1:45040)
0808 18:22:57.618839 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:57.618914 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.188:37769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151}
I20260808 18:22:57.619668 31162 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [term 1 LEARNER]: Refusing update from remote peer e9d826ba0df445bfb8cc41e8dc8c9f18: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260808 18:22:57.620383 31107 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 38137 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260808 18:22:57.621246 31106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:57.621003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:37769 (local address 127.0.0.1:45042)
0808 18:22:57.621126 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:57.621172 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.188:37769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260808 18:22:57.621865 30466 consensus_peers.cc:487] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 -> Peer 9d31cd09bc7843f4854913a5b460ba84 (127.23.72.188:37769): Couldn't send request to peer 9d31cd09bc7843f4854913a5b460ba84. Status: Network error: Client connection negotiation failed: client connection to 127.23.72.188:37769: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 18:22:57.622498 31107 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 11d877a7b3c344f89673448718d22e9a (127.23.72.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } peers { permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 38137 } attrs { promote: false } } peers { permanent_uuid: "9d31cd09bc7843f4854913a5b460ba84" member_type: NON_VOTER last_known_addr { host: "127.23.72.188" port: 37769 } attrs { promote: true } } }
I20260808 18:22:57.623896 31162 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 11d877a7b3c344f89673448718d22e9a (127.23.72.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } peers { permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 38137 } attrs { promote: false } } peers { permanent_uuid: "9d31cd09bc7843f4854913a5b460ba84" member_type: NON_VOTER last_known_addr { host: "127.23.72.188" port: 37769 } attrs { promote: true } } }
I20260808 18:22:57.625344 31315 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } peers { permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 38137 } attrs { promote: false } } peers { permanent_uuid: "9d31cd09bc7843f4854913a5b460ba84" member_type: NON_VOTER last_known_addr { host: "127.23.72.188" port: 37769 } attrs { promote: true } } }
I20260808 18:22:57.625473 31315 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:22:57.626569 31107 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } peers { permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 38137 } attrs { promote: false } } peers { permanent_uuid: "9d31cd09bc7843f4854913a5b460ba84" member_type: NON_VOTER last_known_addr { host: "127.23.72.188" port: 37769 } attrs { promote: true } } }
I20260808 18:22:57.626683 31107 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:57.626873 31107 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } peers { permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 38137 } attrs { promote: false } } peers { permanent_uuid: "9d31cd09bc7843f4854913a5b460ba84" member_type: NON_VOTER last_known_addr { host: "127.23.72.188" port: 37769 } attrs { promote: true } } }
I20260808 18:22:57.626962 31107 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:57.628593 30526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:57.618720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49400 (local address 127.23.72.190:36587)
0808 18:22:57.619153 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:57.619158 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:57.619211 (+ 53us) server_negotiation.cc:407] Connection header received
0808 18:22:57.619389 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:57.619392 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:57.619456 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:57.619508 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:57.620193 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.620982 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.622288 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.622598 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.622679 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:57.624290 (+ 1611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:57.624294 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:57.624297 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:57.624323 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:57.626288 (+ 1965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:57.627150 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:57.627155 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:57.627158 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:57.627253 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:57.627617 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:57.627621 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:57.627622 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:57.627830 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:57.627949 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:57.628328 (+ 379us) server_negotiation.cc:299] Negotiation successful
0808 18:22:57.628458 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380}
I20260808 18:22:57.629246 31326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:57.618713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:36587 (local address 127.0.0.1:49400)
0808 18:22:57.619122 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:57.619137 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:57.619287 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:57.619651 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:57.619653 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:57.619954 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:57.620060 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:57.620067 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.621169 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.621172 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:57.622156 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:57.622167 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.622931 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.622935 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:57.623019 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:57.623613 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:57.623628 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:57.624132 (+ 504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:57.626487 (+ 2355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:57.626500 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:57.626505 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:57.626935 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:57.627371 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:57.627374 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:57.627375 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:57.627500 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:57.628024 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:57.628027 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:57.628139 (+ 112us) client_negotiation.cc:769] Sending connection context
0808 18:22:57.628913 (+ 774us) client_negotiation.cc:240] Negotiation successful
0808 18:22:57.629065 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":95,"threads_started":1}
I20260808 18:22:57.629874 30507 tablet_copy_service.cc:140] P e9d826ba0df445bfb8cc41e8dc8c9f18: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 9d31cd09bc7843f4854913a5b460ba84 ({username='oryx', principal='oryx/127.23.72.188@KRBTEST.COM'} at 127.0.0.1:49400)
I20260808 18:22:57.629967 30507 tablet_copy_service.cc:161] P e9d826ba0df445bfb8cc41e8dc8c9f18: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 9d31cd09bc7843f4854913a5b460ba84 at {username='oryx', principal='oryx/127.23.72.188@KRBTEST.COM'} at 127.0.0.1:49400: session id = 9d31cd09bc7843f4854913a5b460ba84-00000000000000000000000000000000
I20260808 18:22:57.631012 30507 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 18:22:57.632247 31122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 18:22:57.634616 31122 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84: tablet copy: Starting download of 0 data blocks...
I20260808 18:22:57.634804 31122 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84: tablet copy: Starting download of 1 WAL segments...
I20260808 18:22:57.636327 31122 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 18:22:57.638015 30507 tablet_copy_service.cc:342] P e9d826ba0df445bfb8cc41e8dc8c9f18: Request end of tablet copy session 9d31cd09bc7843f4854913a5b460ba84-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.23.72.188@KRBTEST.COM'} at 127.0.0.1:49400
I20260808 18:22:57.638096 30507 tablet_copy_service.cc:433] P e9d826ba0df445bfb8cc41e8dc8c9f18: ending tablet copy session 9d31cd09bc7843f4854913a5b460ba84-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 9d31cd09bc7843f4854913a5b460ba84
I20260808 18:22:57.645283 31122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:57.647225 31328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:57.647302 31331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:57.647411 31122 server_base.cc:1034] running on GCE node
W20260808 18:22:57.647702 31329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:57.647899 31122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:57.648165 31122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:57.649314 31122 hybrid_clock.cc:648] HybridClock initialized: now 1786213377649301 us; error 30 us; skew 500 ppm
Aug 08 18:22:57 dist-test-slave-5k54 krb5kdc[30435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213377, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.72.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:57.650898 31122 init.cc:377] Logged in from keytab as oryx/127.23.72.188@KRBTEST.COM (short username oryx)
I20260808 18:22:57.651324 31122 webserver.cc:460] Webserver started at http://127.23.72.188:41911/ using document root <none> and password file <none>
I20260808 18:22:57.651530 31122 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:57.651576 31122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:57.652817 31122 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 18:22:57.653779 31340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:57.654052 31122 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 18:22:57.654122 31122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-2/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-2/wal
uuid: "9d31cd09bc7843f4854913a5b460ba84"
format_stamp: "Formatted at 2026-08-08 18:22:57 on dist-test-slave-5k54"
server_key: "b8147d5581a5369cb479599d0a7df80e"
server_key_iv: "689255d02cc0d49d02de848000caa3a2"
server_key_version: "encryptionkey@0"
I20260808 18:22:57.654193 31122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:57.662180 31122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:57.664352 31122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:57.669305 31122 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.188:37769
I20260808 18:22:57.669615 31398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.188:37769 every 8 connection(s)
I20260808 18:22:57.670056 31122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/master-2/data/info.pb
I20260808 18:22:57.671983 31399 sys_catalog.cc:263] Verifying existing consensus state
I20260808 18:22:57.672999 31399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84: Bootstrap starting.
I20260808 18:22:57.684257 31399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 18:22:57.684628 31399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84: Bootstrap complete.
I20260808 18:22:57.685362 31399 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } peers { permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 38137 } attrs { promote: false } } peers { permanent_uuid: "9d31cd09bc7843f4854913a5b460ba84" member_type: NON_VOTER last_known_addr { host: "127.23.72.188" port: 37769 } attrs { promote: true } }
I20260808 18:22:57.685510 31399 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9d31cd09bc7843f4854913a5b460ba84, State: Initialized, Role: LEARNER
I20260808 18:22:57.685622 31399 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } peers { permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 38137 } attrs { promote: false } } peers { permanent_uuid: "9d31cd09bc7843f4854913a5b460ba84" member_type: NON_VOTER last_known_addr { host: "127.23.72.188" port: 37769 } attrs { promote: true } }
I20260808 18:22:57.686079 31399 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:22:57.686041 31402 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } peers { permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 38137 } attrs { promote: false } } peers { permanent_uuid: "9d31cd09bc7843f4854913a5b460ba84" member_type: NON_VOTER last_known_addr { host: "127.23.72.188" port: 37769 } attrs { promote: true } } }
I20260808 18:22:57.686635 31402 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [sys.catalog]: This master's current role is: LEARNER
I20260808 18:22:57.689630 31414 catalog_manager.cc:1259] Loaded cluster ID: bf3f1e45a3cf47969ef2d1762c680e0e
I20260808 18:22:57.689680 31414 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84: loading cluster ID for follower catalog manager: success
I20260808 18:22:57.691931 31414 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84: acquiring CA information for follower catalog manager: success
I20260808 18:22:57.693337 31414 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 18:22:57.708823 30844 heartbeater.cc:503] Master 127.23.72.190:36587 was elected leader, sending a full tablet report...
W20260808 18:22:57.727300 31313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:57.727587 31313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:57.727682 31313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:57.731716 31313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:57.731767 31313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:57.731859 31313 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.129
I20260808 18:22:57.737025 31313 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:35011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.129.keytab
--principal=oryx/127.23.72.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.129:38285
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.72.129
--webserver_port=43901
--tserver_master_addrs=127.23.72.190:36587,127.23.72.189:38137,127.23.72.188:37769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31313
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.129
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:57.738013 31313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:57.738987 31313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:57.748732 31420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:57.748780 31423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:57.748791 31421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:57.749063 31313 server_base.cc:1034] running on GCE node
I20260808 18:22:57.749428 31313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:57.749989 31313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:57.751139 31313 hybrid_clock.cc:648] HybridClock initialized: now 1786213377751121 us; error 28 us; skew 500 ppm
Aug 08 18:22:57 dist-test-slave-5k54 krb5kdc[30435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213377, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.72.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:57.753993 31313 init.cc:377] Logged in from keytab as oryx/127.23.72.129@KRBTEST.COM (short username oryx)
I20260808 18:22:57.754882 31313 webserver.cc:460] Webserver started at http://127.23.72.129:43901/ using document root <none> and password file <none>
I20260808 18:22:57.755460 31313 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:57.755578 31313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:57.759464 31313 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:22:57.761545 31430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:57.762670 31313 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:22:57.762796 31313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-0/wal
uuid: "0c00472449774db482dac0a3c10272ed"
format_stamp: "Formatted at 2026-08-08 18:22:56 on dist-test-slave-5k54"
server_key: "0cb2368f4810873075043f295869d2b5"
server_key_iv: "d49d2250ec3080345c9876cb6a0a5585"
server_key_version: "encryptionkey@0"
I20260808 18:22:57.763322 31313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:57.782697 31313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:57.786055 31313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:57.786278 31313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:57.786993 31313 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:22:57.787923 31313 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:22:57.787995 31313 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:57.788065 31313 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:22:57.788116 31313 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:57.797052 31313 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.129:38285
I20260808 18:22:57.797091 31554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.129:38285 every 8 connection(s)
I20260808 18:22:57.798003 31313 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
I20260808 18:22:57.800113 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 31313
I20260808 18:22:57.800333 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 30701
I20260808 18:22:57.809943 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.130:33107
--local_ip_for_outbound_sockets=127.23.72.130
--tserver_master_addrs=127.23.72.190:36587
--webserver_port=34621
--webserver_interface=127.23.72.130
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.130.keytab
--principal=oryx/127.23.72.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:35011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.23.72.190:36587,127.23.72.189:38137
--tserver_master_addrs=127.23.72.190:36587,127.23.72.189:38137,127.23.72.188:37769 with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 18:22:57 dist-test-slave-5k54 krb5kdc[30435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213377, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.72.129@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM
Aug 08 18:22:57 dist-test-slave-5k54 krb5kdc[30435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213377, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.72.129@KRBTEST.COM for oryx/127.23.72.189@KRBTEST.COM
Aug 08 18:22:57 dist-test-slave-5k54 krb5kdc[30435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213377, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.72.129@KRBTEST.COM for oryx/127.23.72.188@KRBTEST.COM
I20260808 18:22:57.816596 31288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:57.799942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:57007 (local address 127.23.72.189:38137)
0808 18:22:57.800236 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:57.800241 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:57.801936 (+ 1695us) server_negotiation.cc:407] Connection header received
0808 18:22:57.802942 (+ 1006us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:57.802949 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:57.803027 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:57.803142 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:57.805013 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.805551 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.807157 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.807532 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.807610 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:57.811751 (+ 4141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:57.811758 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:57.811763 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:57.811818 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:57.813825 (+ 2007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:57.814417 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:57.814424 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:57.814429 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:57.814528 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:57.814912 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:57.814919 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:57.814924 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:57.815235 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:57.815330 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:57.816197 (+ 867us) server_negotiation.cc:299] Negotiation successful
0808 18:22:57.816351 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211}
I20260808 18:22:57.816610 30526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:57.800106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:37875 (local address 127.23.72.190:36587)
0808 18:22:57.800334 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:57.800338 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:57.801992 (+ 1654us) server_negotiation.cc:407] Connection header received
0808 18:22:57.803306 (+ 1314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:57.803313 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:57.803380 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:57.803501 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:57.805006 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.806022 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.807156 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.807532 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.807610 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:57.810818 (+ 3208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:57.810822 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:57.810824 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:57.810853 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:57.813433 (+ 2580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:57.814208 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:57.814211 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:57.814213 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:57.814371 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:57.814840 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:57.814843 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:57.814844 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:57.815447 (+ 603us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:57.815551 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:57.816280 (+ 729us) server_negotiation.cc:299] Negotiation successful
0808 18:22:57.816417 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154}
I20260808 18:22:57.818339 31563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:57.801358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.189:38137 (local address 127.23.72.129:57007)
0808 18:22:57.801841 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:57.801856 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:57.802691 (+ 835us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:57.803286 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:57.803292 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:57.803873 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:57.804826 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:57.804846 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.805711 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.805714 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:57.807013 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:57.807024 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.807810 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.807814 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:57.809249 (+ 1435us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:57.809797 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:57.809816 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:57.811555 (+ 1739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:57.813970 (+ 2415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:57.813977 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:57.813982 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:57.814290 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:57.814662 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:57.814668 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:57.814674 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:57.814793 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:57.815454 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:57.815464 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:57.815851 (+ 387us) client_negotiation.cc:769] Sending connection context
0808 18:22:57.816788 (+ 937us) client_negotiation.cc:240] Negotiation successful
0808 18:22:57.817315 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"mutex_wait_us":1453,"spinlock_wait_cycles":26112,"thread_start_us":111,"threads_started":1}
I20260808 18:22:57.818337 31562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:57.800350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:36587 (local address 127.23.72.129:37875)
0808 18:22:57.801751 (+ 1401us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:57.801810 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:57.802691 (+ 881us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:57.803512 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:57.803515 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:57.803873 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:57.804826 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:57.804846 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.806153 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.806156 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:57.807016 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:57.807024 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.807810 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.807814 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:57.808136 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:57.808774 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:57.808795 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:57.810636 (+ 1841us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:57.813567 (+ 2931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:57.813575 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:57.813594 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:57.814046 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:57.814514 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:57.814517 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:57.814519 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:57.814728 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:57.815686 (+ 958us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:57.815693 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:57.816063 (+ 370us) client_negotiation.cc:769] Sending connection context
0808 18:22:57.816780 (+ 717us) client_negotiation.cc:240] Negotiation successful
0808 18:22:57.817194 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1192,"mutex_wait_us":238,"spinlock_wait_cycles":43776,"thread_start_us":142,"threads_started":1}
I20260808 18:22:57.820077 31560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:57.800892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:50293 (local address 127.23.72.188:37769)
0808 18:22:57.801426 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:57.801433 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:57.807032 (+ 5599us) server_negotiation.cc:407] Connection header received
0808 18:22:57.807244 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:57.807251 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:57.807318 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:57.807390 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:57.808194 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.809918 (+ 1724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.811673 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.812169 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.812256 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:57.814851 (+ 2595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:57.814855 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:57.814857 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:57.814883 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:57.816661 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:57.817624 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:57.817628 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:57.817630 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:57.817717 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:57.818019 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:57.818023 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:57.818024 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:57.818225 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:57.818301 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:57.819805 (+ 1504us) server_negotiation.cc:299] Negotiation successful
0808 18:22:57.819951 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":322,"threads_started":1}
I20260808 18:22:57.820242 31556 heartbeater.cc:347] Connected to a master server at 127.23.72.190:36587
I20260808 18:22:57.820255 31557 heartbeater.cc:347] Connected to a master server at 127.23.72.189:38137
I20260808 18:22:57.820499 31556 heartbeater.cc:464] Registering TS with master...
I20260808 18:22:57.820503 31557 heartbeater.cc:464] Registering TS with master...
I20260808 18:22:57.821036 31556 heartbeater.cc:511] Master 127.23.72.190:36587 requested a full tablet report, sending...
I20260808 18:22:57.821046 31557 heartbeater.cc:511] Master 127.23.72.189:38137 requested a full tablet report, sending...
I20260808 18:22:57.822312 30484 ts_manager.cc:194] Re-registered known tserver with Master: 0c00472449774db482dac0a3c10272ed (127.23.72.129:38285)
I20260808 18:22:57.822650 31153 ts_manager.cc:194] Registered new tserver with Master: 0c00472449774db482dac0a3c10272ed (127.23.72.129:38285)
I20260808 18:22:57.823088 30484 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.72.129@KRBTEST.COM'} at 127.23.72.129:37875
I20260808 18:22:57.823575 31559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:57.800350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:37769 (local address 127.23.72.129:50293)
0808 18:22:57.806930 (+ 6580us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:57.806949 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:57.807110 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:57.807603 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:57.807609 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:57.807893 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:57.808066 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:57.808078 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.810329 (+ 2251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.810332 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:57.811531 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:57.811545 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.812456 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.812460 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:57.812561 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:57.813113 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:57.813131 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:57.814694 (+ 1563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:57.817277 (+ 2583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:57.817279 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:57.817281 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:57.817509 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:57.817819 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:57.817822 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:57.817824 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:57.817917 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:57.818363 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:57.818365 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:57.819615 (+ 1250us) client_negotiation.cc:769] Sending connection context
0808 18:22:57.821623 (+ 2008us) client_negotiation.cc:240] Negotiation successful
0808 18:22:57.821772 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6457,"mutex_wait_us":1229,"thread_start_us":93,"threads_started":1}
I20260808 18:22:57.824363 31555 heartbeater.cc:347] Connected to a master server at 127.23.72.188:37769
I20260808 18:22:57.824468 31555 heartbeater.cc:464] Registering TS with master...
I20260808 18:22:57.824676 31555 heartbeater.cc:511] Master 127.23.72.188:37769 requested a full tablet report, sending...
I20260808 18:22:57.825737 31358 ts_manager.cc:194] Registered new tserver with Master: 0c00472449774db482dac0a3c10272ed (127.23.72.129:38285)
W20260808 18:22:57.940563 31564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:57.940847 31564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:57.940935 31564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:57.945667 31564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:57.945749 31564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:57.945845 31564 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.130
I20260808 18:22:57.951313 31564 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:35011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.130.keytab
--principal=oryx/127.23.72.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.130:33107
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.72.130
--webserver_port=34621
--tserver_master_addrs=127.23.72.190:36587,127.23.72.189:38137,127.23.72.188:37769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31564
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.130
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:57.952294 31564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:57.953253 31564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:57.962826 31572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:57.962883 31570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:57.963023 31564 server_base.cc:1034] running on GCE node
W20260808 18:22:57.962836 31569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:57.963410 30993 heartbeater.cc:503] Master 127.23.72.190:36587 was elected leader, sending a full tablet report...
I20260808 18:22:57.963619 31564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:57.964216 31564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:57.965408 31564 hybrid_clock.cc:648] HybridClock initialized: now 1786213377965391 us; error 42 us; skew 500 ppm
Aug 08 18:22:57 dist-test-slave-5k54 krb5kdc[30435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213377, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.72.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:57.968286 31564 init.cc:377] Logged in from keytab as oryx/127.23.72.130@KRBTEST.COM (short username oryx)
I20260808 18:22:57.969040 31564 webserver.cc:460] Webserver started at http://127.23.72.130:34621/ using document root <none> and password file <none>
I20260808 18:22:57.969529 31564 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:57.969597 31564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:57.973173 31564 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260808 18:22:57.975080 31580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:57.976315 31564 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:22:57.976454 31564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-1/wal
uuid: "f9977432298448988bdb259803c1171b"
format_stamp: "Formatted at 2026-08-08 18:22:56 on dist-test-slave-5k54"
server_key: "3f4bd2ba953dc8accdcaddfaa0beac0c"
server_key_iv: "7f6301dae52f6361552eab98d98dfe31"
server_key_version: "encryptionkey@0"
I20260808 18:22:57.976950 31564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:57.990859 31564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:57.993474 31564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:57.993664 31564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:57.994195 31564 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:22:57.994985 31564 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:22:57.995054 31564 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:57.995121 31564 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:22:57.995154 31564 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:58.000032 31560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:57.995850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45046 (local address 127.23.72.188:37769)
0808 18:22:57.996010 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:57.996015 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:57.996030 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:22:57.996134 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:57.996137 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:57.996201 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:57.996291 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:22:57.996568 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.997121 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.998699 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.999383 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.999546 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:57.999732 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:57.999839 (+ 107us) server_negotiation.cc:299] Negotiation successful
0808 18:22:57.999926 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 18:22:58.000138 31106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:57.995756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:37769 (local address 127.0.0.1:45046)
0808 18:22:57.995916 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:57.995935 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:57.996034 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:57.996319 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:57.996322 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:57.996341 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:22:57.996451 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:57.996456 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.997260 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.997267 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:57.998575 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:57.998590 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:57.999555 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:57.999561 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:57.999691 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:57.999728 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 18:22:57.999827 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 18:22:57.999950 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 18:22:58.000928 31368 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260808 18:22:58.001348 31402 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9d826ba0df445bfb8cc41e8dc8c9f18. Latest consensus state: current_term: 1 leader_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } peers { permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 38137 } attrs { promote: false } } peers { permanent_uuid: "9d31cd09bc7843f4854913a5b460ba84" member_type: NON_VOTER last_known_addr { host: "127.23.72.188" port: 37769 } attrs { promote: true } } }
I20260808 18:22:58.001509 31402 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [sys.catalog]: This master's current role is: LEARNER
I20260808 18:22:58.005170 31564 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.130:33107
I20260808 18:22:58.005194 31703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.130:33107 every 8 connection(s)
I20260808 18:22:58.006131 31564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
I20260808 18:22:58.008858 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 31564
I20260808 18:22:58.009069 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 30850
I20260808 18:22:58.021190 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.131:41097
--local_ip_for_outbound_sockets=127.23.72.131
--tserver_master_addrs=127.23.72.190:36587
--webserver_port=43467
--webserver_interface=127.23.72.131
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.131.keytab
--principal=oryx/127.23.72.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:35011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.23.72.190:36587,127.23.72.189:38137
--tserver_master_addrs=127.23.72.190:36587,127.23.72.189:38137,127.23.72.188:37769 with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 18:22:58 dist-test-slave-5k54 krb5kdc[30435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213377, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.72.130@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM
Aug 08 18:22:58 dist-test-slave-5k54 krb5kdc[30435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213377, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.72.130@KRBTEST.COM for oryx/127.23.72.188@KRBTEST.COM
Aug 08 18:22:58 dist-test-slave-5k54 krb5kdc[30435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213377, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.72.130@KRBTEST.COM for oryx/127.23.72.189@KRBTEST.COM
I20260808 18:22:58.028965 31560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:58.008750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:45661 (local address 127.23.72.188:37769)
0808 18:22:58.008881 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:58.008887 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:58.012623 (+ 3736us) server_negotiation.cc:407] Connection header received
0808 18:22:58.013755 (+ 1132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:58.013761 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:58.013847 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:58.013952 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:58.016375 (+ 2423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.017188 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.019325 (+ 2137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.019700 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.019777 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:58.024359 (+ 4582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:58.024366 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:58.024369 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:58.024400 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:58.026288 (+ 1888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:58.027058 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:58.027061 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:58.027064 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:58.027148 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:58.027604 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:58.027607 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:58.027609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:58.027821 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:58.027908 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:58.028671 (+ 763us) server_negotiation.cc:299] Negotiation successful
0808 18:22:58.028795 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 18:22:58.028965 30526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:58.011114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:58909 (local address 127.23.72.190:36587)
0808 18:22:58.011981 (+ 867us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:58.011988 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:58.012666 (+ 678us) server_negotiation.cc:407] Connection header received
0808 18:22:58.014118 (+ 1452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:58.014125 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:58.014193 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:58.014286 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:58.016406 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.017183 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.019325 (+ 2142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.019700 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.019777 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:58.024233 (+ 4456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:58.024238 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:58.024243 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:58.024276 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:58.026413 (+ 2137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:58.027058 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:58.027062 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:58.027064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:58.027148 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:58.027604 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:58.027607 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:58.027609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:58.027821 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:58.027908 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:58.028671 (+ 763us) server_negotiation.cc:299] Negotiation successful
0808 18:22:58.028795 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221}
I20260808 18:22:58.030136 31708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:58.008323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.189:38137 (local address 127.23.72.130:54863)
0808 18:22:58.014029 (+ 5706us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:58.014043 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:58.014147 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:58.014626 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:58.014639 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:58.015302 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:58.016198 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:58.016221 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.017455 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.017461 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:58.019148 (+ 1687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:58.019162 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.020029 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.020035 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:58.020283 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:58.021064 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:58.021088 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:58.023993 (+ 2905us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:58.026829 (+ 2836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:58.026834 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:58.026837 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:58.027115 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:58.027492 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:58.027498 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:58.027502 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:58.027606 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:58.028172 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:58.028178 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:58.028369 (+ 191us) client_negotiation.cc:769] Sending connection context
0808 18:22:58.028594 (+ 225us) client_negotiation.cc:240] Negotiation successful
0808 18:22:58.028975 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5615,"mutex_wait_us":407,"spinlock_wait_cycles":26240,"thread_start_us":153,"threads_started":1}
I20260808 18:22:58.030133 31709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:58.008322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:36587 (local address 127.23.72.130:58909)
0808 18:22:58.012126 (+ 3804us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:58.012177 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:58.013808 (+ 1631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:58.015852 (+ 2044us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:58.015857 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:58.016044 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:58.016261 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:58.016271 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.017380 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.017383 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:58.019184 (+ 1801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:58.019195 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.019943 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.019946 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:58.020327 (+ 381us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:58.021366 (+ 1039us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:58.021385 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:58.024081 (+ 2696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:58.026518 (+ 2437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:58.026521 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:58.026523 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:58.026938 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:58.027259 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:58.027262 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:58.027264 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:58.027488 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:58.027980 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:58.027987 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:58.028379 (+ 392us) client_negotiation.cc:769] Sending connection context
0808 18:22:58.029105 (+ 726us) client_negotiation.cc:240] Negotiation successful
0808 18:22:58.029262 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3609,"mutex_wait_us":583,"spinlock_wait_cycles":21632,"thread_start_us":125,"threads_started":1}
I20260808 18:22:58.030405 31710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:58.008323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:37769 (local address 127.23.72.130:45661)
0808 18:22:58.011661 (+ 3338us) spinlock_profiling.cc:243] Waited 2.73 ms on lock 0x2444a29c. stack: 00007faab410dc78 00007faab410e1a9 00007faab3cd01d7 00000000005984a6 00000000005984f9 000000000059c488 00007faab4084ac8 00007faab40855c5 00007faaba61c5b1 00007faaba61c620 00007faab411eb33 0000000000618a96 00007faab412ed94 00007faab412d62f 00007faab412ccd8 00007faab87b86b1
0808 18:22:58.012495 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:58.012509 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:58.013530 (+ 1021us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:58.014627 (+ 1097us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:58.014639 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:58.015298 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:58.016221 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:58.016234 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.017428 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.017433 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:58.019184 (+ 1751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:58.019195 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.019929 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.019932 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:58.020329 (+ 397us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:58.021220 (+ 891us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:58.021236 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:58.024221 (+ 2985us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:58.026428 (+ 2207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:58.026436 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:58.026454 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:58.026938 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:58.027259 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:58.027262 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:58.027264 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:58.027488 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:58.027980 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:58.027987 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:58.028378 (+ 391us) client_negotiation.cc:769] Sending connection context
0808 18:22:58.029212 (+ 834us) client_negotiation.cc:240] Negotiation successful
0808 18:22:58.029317 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4068,"mutex_wait_us":567,"spinlock_wait_cycles":6296704,"thread_start_us":3361,"threads_started":1}
I20260808 18:22:58.031512 31705 heartbeater.cc:347] Connected to a master server at 127.23.72.190:36587
I20260808 18:22:58.031543 31704 heartbeater.cc:347] Connected to a master server at 127.23.72.188:37769
I20260808 18:22:58.031744 31705 heartbeater.cc:464] Registering TS with master...
I20260808 18:22:58.031744 31704 heartbeater.cc:464] Registering TS with master...
I20260808 18:22:58.032050 31288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:58.008522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:54863 (local address 127.23.72.189:38137)
0808 18:22:58.008657 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:58.008662 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:58.014115 (+ 5453us) server_negotiation.cc:407] Connection header received
0808 18:22:58.014229 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:58.014235 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:58.014296 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:58.014375 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:58.016381 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.017285 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.019298 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.019742 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.019836 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:58.024191 (+ 4355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:58.024237 (+ 46us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:58.024241 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:58.024275 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:58.026687 (+ 2412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:58.027243 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:58.027248 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:58.027253 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:58.027346 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:58.027740 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:58.027745 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:58.027749 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:58.028011 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:58.028120 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:58.031775 (+ 3655us) server_negotiation.cc:299] Negotiation successful
0808 18:22:58.031904 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 18:22:58.032320 31704 heartbeater.cc:511] Master 127.23.72.188:37769 requested a full tablet report, sending...
I20260808 18:22:58.032658 31706 heartbeater.cc:347] Connected to a master server at 127.23.72.189:38137
I20260808 18:22:58.032734 31706 heartbeater.cc:464] Registering TS with master...
I20260808 18:22:58.032894 31706 heartbeater.cc:511] Master 127.23.72.189:38137 requested a full tablet report, sending...
I20260808 18:22:58.033257 31705 heartbeater.cc:511] Master 127.23.72.190:36587 requested a full tablet report, sending...
I20260808 18:22:58.033515 31358 ts_manager.cc:194] Registered new tserver with Master: f9977432298448988bdb259803c1171b (127.23.72.130:33107)
I20260808 18:22:58.033674 31153 ts_manager.cc:194] Registered new tserver with Master: f9977432298448988bdb259803c1171b (127.23.72.130:33107)
I20260808 18:22:58.033833 30484 ts_manager.cc:194] Re-registered known tserver with Master: f9977432298448988bdb259803c1171b (127.23.72.130:33107)
I20260808 18:22:58.034497 30484 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.72.130@KRBTEST.COM'} at 127.23.72.130:58909
W20260808 18:22:58.137102 31711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:58.137367 31711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:58.137440 31711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:58.141454 31711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:58.141531 31711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:58.141615 31711 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.131
I20260808 18:22:58.146822 31711 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:35011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.131.keytab
--principal=oryx/127.23.72.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.131:41097
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.72.131
--webserver_port=43467
--tserver_master_addrs=127.23.72.190:36587,127.23.72.189:38137,127.23.72.188:37769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31711
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.131
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:58.147928 31711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:58.148950 31711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:58.158522 31716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:58.158532 31717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:58.158532 31720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:58.158823 31711 server_base.cc:1034] running on GCE node
I20260808 18:22:58.159600 31711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:58.160238 31711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:58.161423 31711 hybrid_clock.cc:648] HybridClock initialized: now 1786213378161414 us; error 29 us; skew 500 ppm
Aug 08 18:22:58 dist-test-slave-5k54 krb5kdc[30435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213378, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.72.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:58.164104 31711 init.cc:377] Logged in from keytab as oryx/127.23.72.131@KRBTEST.COM (short username oryx)
I20260808 18:22:58.164891 31711 webserver.cc:460] Webserver started at http://127.23.72.131:43467/ using document root <none> and password file <none>
I20260808 18:22:58.165400 31711 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:58.165473 31711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:58.169437 31711 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:22:58.171311 31728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:58.172531 31711 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:22:58.172650 31711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-2/wal
uuid: "086e23e953ee4ecdb9a68942c3037fd5"
format_stamp: "Formatted at 2026-08-08 18:22:56 on dist-test-slave-5k54"
server_key: "6a397a7d5f8fc29cccf95a7878c18637"
server_key_iv: "4a60a8358a03b0ef09fa66178526e8db"
server_key_version: "encryptionkey@0"
I20260808 18:22:58.173177 31711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:58.199312 31711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:58.202549 31711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:58.202697 31711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:58.203202 31711 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:22:58.204003 31711 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:22:58.204048 31711 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:58.204085 31711 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:22:58.204100 31711 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:58.213156 31711 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.131:41097
I20260808 18:22:58.213181 31856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.131:41097 every 8 connection(s)
I20260808 18:22:58.214097 31711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
I20260808 18:22:58.219146 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 31711
Aug 08 18:22:58 dist-test-slave-5k54 krb5kdc[30435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213378, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.72.131@KRBTEST.COM for oryx/127.23.72.189@KRBTEST.COM
Aug 08 18:22:58 dist-test-slave-5k54 krb5kdc[30435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213378, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.72.131@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM
Aug 08 18:22:58 dist-test-slave-5k54 krb5kdc[30435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213378, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.72.131@KRBTEST.COM for oryx/127.23.72.188@KRBTEST.COM
I20260808 18:22:58.233081 31560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:58.216545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:56261 (local address 127.23.72.188:37769)
0808 18:22:58.216800 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:58.216805 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:58.217448 (+ 643us) server_negotiation.cc:407] Connection header received
0808 18:22:58.218258 (+ 810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:58.218261 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:58.218314 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:58.218367 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:58.220127 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.220941 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.222816 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.223185 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.223280 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:58.228592 (+ 5312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:58.228596 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:58.228598 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:58.228623 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:58.230434 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:58.231122 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:58.231126 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:58.231131 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:58.231236 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:58.231717 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:58.231723 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:58.231725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:58.231968 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:58.232068 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:58.232773 (+ 705us) server_negotiation.cc:299] Negotiation successful
0808 18:22:58.232897 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177}
I20260808 18:22:58.233963 31863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:58.216321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:37769 (local address 127.23.72.131:56261)
0808 18:22:58.217321 (+ 1000us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:58.217333 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:58.218063 (+ 730us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:58.218573 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:58.218580 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:58.219155 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:58.219968 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:58.219981 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.221086 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.221089 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:58.222664 (+ 1575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:58.222675 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.223461 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.223466 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:58.223753 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:58.224618 (+ 865us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:58.224642 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:58.228478 (+ 3836us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:58.230574 (+ 2096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:58.230579 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:58.230593 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:58.230998 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:58.231356 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:58.231359 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:58.231361 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:58.231593 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:58.232115 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:58.232119 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:58.232346 (+ 227us) client_negotiation.cc:769] Sending connection context
0808 18:22:58.232548 (+ 202us) client_negotiation.cc:240] Negotiation successful
0808 18:22:58.233441 (+ 893us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":939,"mutex_wait_us":1363,"spinlock_wait_cycles":91392,"thread_start_us":147,"threads_started":1}
I20260808 18:22:58.234406 31288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:58.216676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:34013 (local address 127.23.72.189:38137)
0808 18:22:58.216885 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:58.216889 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:58.217407 (+ 518us) server_negotiation.cc:407] Connection header received
0808 18:22:58.218233 (+ 826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:58.218236 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:58.218289 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:58.218403 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:58.220127 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.220941 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.222786 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.223128 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.223220 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:58.227969 (+ 4749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:58.227972 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:58.227974 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:58.228008 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:58.230723 (+ 2715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:58.231182 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:58.231189 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:58.231193 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:58.231283 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:58.231692 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:58.231696 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:58.231697 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:58.231896 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:58.231982 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:58.234097 (+ 2115us) server_negotiation.cc:299] Negotiation successful
0808 18:22:58.234209 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144}
I20260808 18:22:58.233970 31864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:58.216317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.189:38137 (local address 127.23.72.131:34013)
0808 18:22:58.217309 (+ 992us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:58.217324 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:58.218044 (+ 720us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:58.218571 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:58.218578 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:58.219135 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:58.219963 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:58.219981 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.221086 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.221089 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:58.222668 (+ 1579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:58.222674 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.223384 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.223388 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:58.223751 (+ 363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:58.225048 (+ 1297us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:58.225063 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:58.227799 (+ 2736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:58.230831 (+ 3032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:58.230834 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:58.230836 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:58.231075 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:58.231396 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:58.231399 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:58.231403 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:58.231579 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:58.232031 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:58.232037 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:58.232352 (+ 315us) client_negotiation.cc:769] Sending connection context
0808 18:22:58.232547 (+ 195us) client_negotiation.cc:240] Negotiation successful
0808 18:22:58.232897 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":889,"mutex_wait_us":641,"spinlock_wait_cycles":36864,"thread_start_us":134,"threads_started":1}
I20260808 18:22:58.235641 31860 heartbeater.cc:347] Connected to a master server at 127.23.72.189:38137
I20260808 18:22:58.235644 31857 heartbeater.cc:347] Connected to a master server at 127.23.72.188:37769
I20260808 18:22:58.235878 31860 heartbeater.cc:464] Registering TS with master...
I20260808 18:22:58.235879 31857 heartbeater.cc:464] Registering TS with master...
I20260808 18:22:58.236397 30526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:58.215851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:49629 (local address 127.23.72.190:36587)
0808 18:22:58.215979 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:58.215984 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:58.217297 (+ 1313us) server_negotiation.cc:407] Connection header received
0808 18:22:58.218258 (+ 961us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:58.218261 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:58.218314 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:58.218367 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:58.220143 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.220883 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.222791 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.223140 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.223214 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:58.228706 (+ 5492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:58.228710 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:58.228715 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:58.228748 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:58.230575 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:58.234665 (+ 4090us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:58.234669 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:58.234671 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:58.234764 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:58.235365 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:58.235368 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:58.235370 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:58.235589 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:58.235679 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:58.236138 (+ 459us) server_negotiation.cc:299] Negotiation successful
0808 18:22:58.236250 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 18:22:58.236496 31857 heartbeater.cc:511] Master 127.23.72.188:37769 requested a full tablet report, sending...
I20260808 18:22:58.236825 31869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:58.221448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49412 (local address 127.23.72.190:36587)
0808 18:22:58.222766 (+ 1318us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:58.222781 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:58.223759 (+ 978us) server_negotiation.cc:407] Connection header received
0808 18:22:58.223804 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:58.223807 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:58.223857 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:58.223952 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:58.225879 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.226710 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.228143 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.228768 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.228872 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:58.230604 (+ 1732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:58.230608 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:58.230610 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:58.230636 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:58.232445 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:58.232939 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:58.232943 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:58.232944 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:58.233025 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:58.233577 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:58.233580 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:58.233582 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:58.233783 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:58.233918 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:58.234551 (+ 633us) server_negotiation.cc:299] Negotiation successful
0808 18:22:58.236615 (+ 2064us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1231,"thread_start_us":67,"threads_started":1}
I20260808 18:22:58.236497 31860 heartbeater.cc:511] Master 127.23.72.189:38137 requested a full tablet report, sending...
I20260808 18:22:58.237290 31862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:58.216302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:36587 (local address 127.23.72.131:49629)
0808 18:22:58.217129 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:58.217180 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:58.218063 (+ 883us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:58.218573 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:58.218580 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:58.219155 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:58.219964 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:58.219983 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.221058 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.221060 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:58.222647 (+ 1587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:58.222660 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.223384 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.223388 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:58.223852 (+ 464us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:58.224870 (+ 1018us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:58.224886 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:58.227347 (+ 2461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:58.230707 (+ 3360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:58.230711 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:58.230713 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:58.234442 (+ 3729us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:58.235155 (+ 713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:58.235158 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:58.235160 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:58.235252 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:58.235789 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:58.235792 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:58.235888 (+ 96us) client_negotiation.cc:769] Sending connection context
0808 18:22:58.236739 (+ 851us) client_negotiation.cc:240] Negotiation successful
0808 18:22:58.237086 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"mutex_wait_us":415,"spinlock_wait_cycles":69248,"thread_start_us":137,"threads_started":1}
I20260808 18:22:58.237957 31358 ts_manager.cc:194] Registered new tserver with Master: 086e23e953ee4ecdb9a68942c3037fd5 (127.23.72.131:41097)
I20260808 18:22:58.237984 31858 heartbeater.cc:347] Connected to a master server at 127.23.72.190:36587
I20260808 18:22:58.238047 31858 heartbeater.cc:464] Registering TS with master...
I20260808 18:22:58.238196 31858 heartbeater.cc:511] Master 127.23.72.190:36587 requested a full tablet report, sending...
I20260808 18:22:58.238303 31153 ts_manager.cc:194] Registered new tserver with Master: 086e23e953ee4ecdb9a68942c3037fd5 (127.23.72.131:41097)
I20260808 18:22:58.238708 30484 ts_manager.cc:194] Re-registered known tserver with Master: 086e23e953ee4ecdb9a68942c3037fd5 (127.23.72.131:41097)
I20260808 18:22:58.239414 30484 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.72.131@KRBTEST.COM'} at 127.23.72.131:49629
I20260808 18:22:58.239797 31870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:58.221874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40900 (local address 127.23.72.189:38137)
0808 18:22:58.222922 (+ 1048us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:58.222926 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:58.223331 (+ 405us) server_negotiation.cc:407] Connection header received
0808 18:22:58.224961 (+ 1630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:58.224965 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:58.225011 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:58.225098 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:58.225931 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.226711 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.228143 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.228856 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.228946 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:58.235926 (+ 6980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:58.235932 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:58.235935 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:58.235961 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:58.237688 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:58.238159 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:58.238162 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:58.238164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:58.238246 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:58.238563 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:58.238568 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:58.238570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:58.238767 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:58.238883 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:58.239467 (+ 584us) server_negotiation.cc:299] Negotiation successful
0808 18:22:58.239612 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":974,"thread_start_us":93,"threads_started":1}
I20260808 18:22:58.241169 30484 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49412:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 18:22:58.244645 31874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:58.223943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45056 (local address 127.23.72.188:37769)
0808 18:22:58.224507 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:58.224512 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:58.224530 (+ 18us) server_negotiation.cc:407] Connection header received
0808 18:22:58.224666 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:58.224669 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:58.224719 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:58.224823 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:58.226706 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.231154 (+ 4448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.232304 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.232628 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.232715 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:58.241462 (+ 8747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:58.241465 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:58.241467 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:58.241489 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:58.242909 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:58.243355 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:58.243357 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:58.243359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:58.243486 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:58.243800 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:58.243804 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:58.243808 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:58.244047 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:58.244165 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:58.244386 (+ 221us) server_negotiation.cc:299] Negotiation successful
0808 18:22:58.244503 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":74,"threads_started":1}
W20260808 18:22:58.244727 30484 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:22:58.247251 31327 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } peers { permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 38137 } attrs { promote: false } } peers { permanent_uuid: "9d31cd09bc7843f4854913a5b460ba84" member_type: NON_VOTER last_known_addr { host: "127.23.72.188" port: 37769 } attrs { promote: true } } }
I20260808 18:22:58.247406 31327 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:58.250445 31877 raft_consensus.cc:1050] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18: attempting to promote NON_VOTER 9d31cd09bc7843f4854913a5b460ba84 to VOTER
I20260808 18:22:58.250770 31877 consensus_queue.cc:237] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } peers { permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 38137 } attrs { promote: false } } peers { permanent_uuid: "9d31cd09bc7843f4854913a5b460ba84" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 37769 } attrs { promote: false } }
I20260808 18:22:58.253737 31368 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [term 1 LEARNER]: Refusing update from remote peer e9d826ba0df445bfb8cc41e8dc8c9f18: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 18:22:58.253772 31162 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [term 1 FOLLOWER]: Refusing update from remote peer e9d826ba0df445bfb8cc41e8dc8c9f18: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 18:22:58.254220 31880 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 38137 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260808 18:22:58.254371 31877 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d31cd09bc7843f4854913a5b460ba84" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 37769 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260808 18:22:58.256233 31877 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 9d31cd09bc7843f4854913a5b460ba84 (127.23.72.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } peers { permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 38137 } attrs { promote: false } } peers { permanent_uuid: "9d31cd09bc7843f4854913a5b460ba84" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 37769 } attrs { promote: false } } }
I20260808 18:22:58.256633 31162 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 9d31cd09bc7843f4854913a5b460ba84 (127.23.72.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } peers { permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 38137 } attrs { promote: false } } peers { permanent_uuid: "9d31cd09bc7843f4854913a5b460ba84" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 37769 } attrs { promote: false } } }
I20260808 18:22:58.256711 31368 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 9d31cd09bc7843f4854913a5b460ba84 (127.23.72.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } peers { permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 38137 } attrs { promote: false } } peers { permanent_uuid: "9d31cd09bc7843f4854913a5b460ba84" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 37769 } attrs { promote: false } } }
I20260808 18:22:58.257654 31327 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } peers { permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 38137 } attrs { promote: false } } peers { permanent_uuid: "9d31cd09bc7843f4854913a5b460ba84" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 37769 } attrs { promote: false } } }
I20260808 18:22:58.257748 31327 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:58.258229 31877 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } peers { permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 38137 } attrs { promote: false } } peers { permanent_uuid: "9d31cd09bc7843f4854913a5b460ba84" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 37769 } attrs { promote: false } } }
I20260808 18:22:58.258283 31327 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } peers { permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 38137 } attrs { promote: false } } peers { permanent_uuid: "9d31cd09bc7843f4854913a5b460ba84" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 37769 } attrs { promote: false } } }
I20260808 18:22:58.258363 31877 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:58.258414 31327 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18 [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:58.259083 31402 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } peers { permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 38137 } attrs { promote: false } } peers { permanent_uuid: "9d31cd09bc7843f4854913a5b460ba84" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 37769 } attrs { promote: false } } }
I20260808 18:22:58.259209 31402 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:22:58.261171 31889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } peers { permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 38137 } attrs { promote: false } } peers { permanent_uuid: "9d31cd09bc7843f4854913a5b460ba84" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 37769 } attrs { promote: false } } }
I20260808 18:22:58.261325 31889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:22:58.265939 31891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:58.260370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.130:33107 (local address 127.0.0.1:45400)
0808 18:22:58.260630 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:58.260644 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:58.260729 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:58.262222 (+ 1493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:58.262225 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:58.262244 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:22:58.262332 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:58.262339 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.263614 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.263618 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:58.264622 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:58.264631 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.265548 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.265552 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:58.265634 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:58.265656 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 18:22:58.265719 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 18:22:58.265799 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":51,"thread_start_us":129,"threads_started":1}
I20260808 18:22:58.265964 31893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:58.260872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45400 (local address 127.23.72.130:33107)
0808 18:22:58.261741 (+ 869us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:58.261753 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:58.261775 (+ 22us) server_negotiation.cc:407] Connection header received
0808 18:22:58.261882 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:58.261887 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:58.262079 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:58.262235 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:22:58.262431 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.263525 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.264753 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.265424 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.265549 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:58.265697 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:58.265750 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 18:22:58.265836 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":765,"thread_start_us":118,"threads_started":1}
I20260808 18:22:58.266327 31106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:58.260093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:38285 (local address 127.0.0.1:57772)
0808 18:22:58.260221 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:58.260237 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:58.260317 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:58.261882 (+ 1565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:58.261885 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:58.261905 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:22:58.262002 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:58.262008 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.263547 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.263552 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:58.264809 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:58.264821 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.265965 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.265968 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:58.266054 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:58.266074 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 18:22:58.266125 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 18:22:58.266193 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":57}
I20260808 18:22:58.266502 31892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:58.260711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.131:41097 (local address 127.0.0.1:35196)
0808 18:22:58.260979 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:58.260991 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:58.261221 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:58.262533 (+ 1312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:58.262535 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:58.262551 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:22:58.262626 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:58.262634 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.263600 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.263603 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:58.264843 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:58.264850 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.266133 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.266136 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:58.266217 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:58.266236 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 18:22:58.266297 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 18:22:58.266371 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"mutex_wait_us":52,"thread_start_us":119,"threads_started":1}
I20260808 18:22:58.266893 31894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:58.261125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35196 (local address 127.23.72.131:41097)
0808 18:22:58.262110 (+ 985us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:58.262119 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:58.262135 (+ 16us) server_negotiation.cc:407] Connection header received
0808 18:22:58.262222 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:58.262227 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:58.262411 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:58.262543 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:22:58.262713 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.263476 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.264966 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.265823 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.265941 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:58.266072 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:58.266668 (+ 596us) server_negotiation.cc:299] Negotiation successful
0808 18:22:58.266761 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":905,"thread_start_us":98,"threads_started":1}
I20260808 18:22:58.267606 31890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:58.260351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57772 (local address 127.23.72.129:38285)
0808 18:22:58.261393 (+ 1042us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:58.261415 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:58.261438 (+ 23us) server_negotiation.cc:407] Connection header received
0808 18:22:58.261514 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:58.261518 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:58.261756 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:58.261884 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:22:58.262145 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.263384 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.264966 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.265823 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.267181 (+ 1358us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:58.267313 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:58.267358 (+ 45us) server_negotiation.cc:299] Negotiation successful
0808 18:22:58.267468 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":127,"threads_started":1}
I20260808 18:22:58.269161 31632 tablet_service.cc:1467] Processing CreateTablet for tablet a4579a8a377344539f5952bbe1aa10a7 (DEFAULT_TABLE table=test-table [id=cbf80ddbb5f048ccad382f0153f5d6ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:22:58.269587 31782 tablet_service.cc:1467] Processing CreateTablet for tablet a4579a8a377344539f5952bbe1aa10a7 (DEFAULT_TABLE table=test-table [id=cbf80ddbb5f048ccad382f0153f5d6ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:22:58.269914 31481 tablet_service.cc:1467] Processing CreateTablet for tablet a4579a8a377344539f5952bbe1aa10a7 (DEFAULT_TABLE table=test-table [id=cbf80ddbb5f048ccad382f0153f5d6ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:22:58.270305 31632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4579a8a377344539f5952bbe1aa10a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:58.270581 31782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4579a8a377344539f5952bbe1aa10a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:58.270829 31481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4579a8a377344539f5952bbe1aa10a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:58.277325 31895 tablet_bootstrap.cc:492] T a4579a8a377344539f5952bbe1aa10a7 P 0c00472449774db482dac0a3c10272ed: Bootstrap starting.
I20260808 18:22:58.278563 31897 tablet_bootstrap.cc:492] T a4579a8a377344539f5952bbe1aa10a7 P f9977432298448988bdb259803c1171b: Bootstrap starting.
I20260808 18:22:58.279734 31895 tablet_bootstrap.cc:654] T a4579a8a377344539f5952bbe1aa10a7 P 0c00472449774db482dac0a3c10272ed: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:58.279740 31896 tablet_bootstrap.cc:492] T a4579a8a377344539f5952bbe1aa10a7 P 086e23e953ee4ecdb9a68942c3037fd5: Bootstrap starting.
I20260808 18:22:58.280452 31895 log.cc:824] T a4579a8a377344539f5952bbe1aa10a7 P 0c00472449774db482dac0a3c10272ed: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:58.280656 31897 tablet_bootstrap.cc:654] T a4579a8a377344539f5952bbe1aa10a7 P f9977432298448988bdb259803c1171b: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:58.281432 31897 log.cc:824] T a4579a8a377344539f5952bbe1aa10a7 P f9977432298448988bdb259803c1171b: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:58.281780 31896 tablet_bootstrap.cc:654] T a4579a8a377344539f5952bbe1aa10a7 P 086e23e953ee4ecdb9a68942c3037fd5: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:58.282398 31895 tablet_bootstrap.cc:492] T a4579a8a377344539f5952bbe1aa10a7 P 0c00472449774db482dac0a3c10272ed: No bootstrap required, opened a new log
I20260808 18:22:58.282514 31896 log.cc:824] T a4579a8a377344539f5952bbe1aa10a7 P 086e23e953ee4ecdb9a68942c3037fd5: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:58.282611 31895 ts_tablet_manager.cc:1397] T a4579a8a377344539f5952bbe1aa10a7 P 0c00472449774db482dac0a3c10272ed: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 18:22:58.283392 31897 tablet_bootstrap.cc:492] T a4579a8a377344539f5952bbe1aa10a7 P f9977432298448988bdb259803c1171b: No bootstrap required, opened a new log
I20260808 18:22:58.283586 31897 ts_tablet_manager.cc:1397] T a4579a8a377344539f5952bbe1aa10a7 P f9977432298448988bdb259803c1171b: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260808 18:22:58.284372 31896 tablet_bootstrap.cc:492] T a4579a8a377344539f5952bbe1aa10a7 P 086e23e953ee4ecdb9a68942c3037fd5: No bootstrap required, opened a new log
I20260808 18:22:58.284545 31896 ts_tablet_manager.cc:1397] T a4579a8a377344539f5952bbe1aa10a7 P 086e23e953ee4ecdb9a68942c3037fd5: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 18:22:58.286900 31895 raft_consensus.cc:348] T a4579a8a377344539f5952bbe1aa10a7 P 0c00472449774db482dac0a3c10272ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c00472449774db482dac0a3c10272ed" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 38285 } } peers { permanent_uuid: "f9977432298448988bdb259803c1171b" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 33107 } } peers { permanent_uuid: "086e23e953ee4ecdb9a68942c3037fd5" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41097 } }
I20260808 18:22:58.287134 31895 raft_consensus.cc:374] T a4579a8a377344539f5952bbe1aa10a7 P 0c00472449774db482dac0a3c10272ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:58.287187 31895 raft_consensus.cc:729] T a4579a8a377344539f5952bbe1aa10a7 P 0c00472449774db482dac0a3c10272ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c00472449774db482dac0a3c10272ed, State: Initialized, Role: FOLLOWER
I20260808 18:22:58.287695 31895 consensus_queue.cc:260] T a4579a8a377344539f5952bbe1aa10a7 P 0c00472449774db482dac0a3c10272ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c00472449774db482dac0a3c10272ed" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 38285 } } peers { permanent_uuid: "f9977432298448988bdb259803c1171b" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 33107 } } peers { permanent_uuid: "086e23e953ee4ecdb9a68942c3037fd5" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41097 } }
I20260808 18:22:58.288403 31556 heartbeater.cc:503] Master 127.23.72.190:36587 was elected leader, sending a full tablet report...
I20260808 18:22:58.289234 31896 raft_consensus.cc:348] T a4579a8a377344539f5952bbe1aa10a7 P 086e23e953ee4ecdb9a68942c3037fd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c00472449774db482dac0a3c10272ed" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 38285 } } peers { permanent_uuid: "f9977432298448988bdb259803c1171b" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 33107 } } peers { permanent_uuid: "086e23e953ee4ecdb9a68942c3037fd5" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41097 } }
I20260808 18:22:58.289505 31896 raft_consensus.cc:374] T a4579a8a377344539f5952bbe1aa10a7 P 086e23e953ee4ecdb9a68942c3037fd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:58.289557 31896 raft_consensus.cc:729] T a4579a8a377344539f5952bbe1aa10a7 P 086e23e953ee4ecdb9a68942c3037fd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 086e23e953ee4ecdb9a68942c3037fd5, State: Initialized, Role: FOLLOWER
I20260808 18:22:58.289988 31896 consensus_queue.cc:260] T a4579a8a377344539f5952bbe1aa10a7 P 086e23e953ee4ecdb9a68942c3037fd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c00472449774db482dac0a3c10272ed" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 38285 } } peers { permanent_uuid: "f9977432298448988bdb259803c1171b" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 33107 } } peers { permanent_uuid: "086e23e953ee4ecdb9a68942c3037fd5" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41097 } }
I20260808 18:22:58.290989 31896 ts_tablet_manager.cc:1428] T a4579a8a377344539f5952bbe1aa10a7 P 086e23e953ee4ecdb9a68942c3037fd5: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 18:22:58.291121 31895 ts_tablet_manager.cc:1428] T a4579a8a377344539f5952bbe1aa10a7 P 0c00472449774db482dac0a3c10272ed: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260808 18:22:58.288220 31897 raft_consensus.cc:348] T a4579a8a377344539f5952bbe1aa10a7 P f9977432298448988bdb259803c1171b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c00472449774db482dac0a3c10272ed" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 38285 } } peers { permanent_uuid: "f9977432298448988bdb259803c1171b" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 33107 } } peers { permanent_uuid: "086e23e953ee4ecdb9a68942c3037fd5" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41097 } }
I20260808 18:22:58.291224 31897 raft_consensus.cc:374] T a4579a8a377344539f5952bbe1aa10a7 P f9977432298448988bdb259803c1171b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:58.291278 31897 raft_consensus.cc:729] T a4579a8a377344539f5952bbe1aa10a7 P f9977432298448988bdb259803c1171b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9977432298448988bdb259803c1171b, State: Initialized, Role: FOLLOWER
I20260808 18:22:58.291337 31858 heartbeater.cc:503] Master 127.23.72.190:36587 was elected leader, sending a full tablet report...
I20260808 18:22:58.291711 31897 consensus_queue.cc:260] T a4579a8a377344539f5952bbe1aa10a7 P f9977432298448988bdb259803c1171b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c00472449774db482dac0a3c10272ed" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 38285 } } peers { permanent_uuid: "f9977432298448988bdb259803c1171b" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 33107 } } peers { permanent_uuid: "086e23e953ee4ecdb9a68942c3037fd5" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41097 } }
I20260808 18:22:58.292591 31897 ts_tablet_manager.cc:1428] T a4579a8a377344539f5952bbe1aa10a7 P f9977432298448988bdb259803c1171b: Time spent starting tablet: real 0.009s user 0.003s sys 0.003s
I20260808 18:22:58.294008 31705 heartbeater.cc:503] Master 127.23.72.190:36587 was elected leader, sending a full tablet report...
W20260808 18:22:58.299933 31558 tablet.cc:2369] T a4579a8a377344539f5952bbe1aa10a7 P 0c00472449774db482dac0a3c10272ed: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 18:22:58.465476 31861 tablet.cc:2369] T a4579a8a377344539f5952bbe1aa10a7 P 086e23e953ee4ecdb9a68942c3037fd5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:22:58.475154 31903 raft_consensus.cc:482] T a4579a8a377344539f5952bbe1aa10a7 P f9977432298448988bdb259803c1171b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:22:58.475382 31903 raft_consensus.cc:504] T a4579a8a377344539f5952bbe1aa10a7 P f9977432298448988bdb259803c1171b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c00472449774db482dac0a3c10272ed" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 38285 } } peers { permanent_uuid: "f9977432298448988bdb259803c1171b" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 33107 } } peers { permanent_uuid: "086e23e953ee4ecdb9a68942c3037fd5" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41097 } }
I20260808 18:22:58.476609 31903 leader_election.cc:288] T a4579a8a377344539f5952bbe1aa10a7 P f9977432298448988bdb259803c1171b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0c00472449774db482dac0a3c10272ed (127.23.72.129:38285), 086e23e953ee4ecdb9a68942c3037fd5 (127.23.72.131:41097)
I20260808 18:22:58.480504 31890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:58.476994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:38889 (local address 127.23.72.129:38285)
0808 18:22:58.477127 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:58.477132 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:58.477253 (+ 121us) server_negotiation.cc:407] Connection header received
0808 18:22:58.477408 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:58.477410 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:58.477475 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:58.477556 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:22:58.477892 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.478508 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.479484 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.479892 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.480017 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:58.480070 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:58.480297 (+ 227us) server_negotiation.cc:299] Negotiation successful
0808 18:22:58.480374 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 18:22:58.480504 31710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:58.476895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:38285 (local address 127.23.72.130:38889)
0808 18:22:58.477185 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:58.477200 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:58.477292 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:58.477586 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:58.477588 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:58.477602 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:22:58.477723 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:58.477728 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.478615 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.478617 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:58.479324 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:58.479332 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.480007 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.480010 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:58.480145 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:58.480164 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 18:22:58.480265 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 18:22:58.480364 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":161}
I20260808 18:22:58.480511 31894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:58.477037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:40397 (local address 127.23.72.131:41097)
0808 18:22:58.477186 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:58.477192 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:58.477240 (+ 48us) server_negotiation.cc:407] Connection header received
0808 18:22:58.477319 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:58.477325 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:58.477383 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:58.477447 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:22:58.477803 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.478335 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.479439 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.479889 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.480033 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:58.480083 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:58.480314 (+ 231us) server_negotiation.cc:299] Negotiation successful
0808 18:22:58.480394 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 18:22:58.480511 31709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:58.476895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.131:41097 (local address 127.23.72.130:40397)
0808 18:22:58.477039 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:58.477058 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:58.477170 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:58.477478 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:58.477481 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:58.477500 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:22:58.477683 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:58.477689 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.478435 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.478439 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:58.479282 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:58.479291 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.480054 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.480058 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:58.480176 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:58.480194 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 18:22:58.480280 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 18:22:58.480388 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":41}
I20260808 18:22:58.481206 31504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4579a8a377344539f5952bbe1aa10a7" candidate_uuid: "f9977432298448988bdb259803c1171b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c00472449774db482dac0a3c10272ed" is_pre_election: true
I20260808 18:22:58.481213 31802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4579a8a377344539f5952bbe1aa10a7" candidate_uuid: "f9977432298448988bdb259803c1171b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "086e23e953ee4ecdb9a68942c3037fd5" is_pre_election: true
I20260808 18:22:58.481516 31802 raft_consensus.cc:2454] T a4579a8a377344539f5952bbe1aa10a7 P 086e23e953ee4ecdb9a68942c3037fd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9977432298448988bdb259803c1171b in term 0.
I20260808 18:22:58.481529 31504 raft_consensus.cc:2454] T a4579a8a377344539f5952bbe1aa10a7 P 0c00472449774db482dac0a3c10272ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9977432298448988bdb259803c1171b in term 0.
I20260808 18:22:58.481984 31581 leader_election.cc:302] T a4579a8a377344539f5952bbe1aa10a7 P f9977432298448988bdb259803c1171b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 086e23e953ee4ecdb9a68942c3037fd5, 0c00472449774db482dac0a3c10272ed, f9977432298448988bdb259803c1171b; no voters:
I20260808 18:22:58.482234 31903 raft_consensus.cc:2781] T a4579a8a377344539f5952bbe1aa10a7 P f9977432298448988bdb259803c1171b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:22:58.482316 31903 raft_consensus.cc:482] T a4579a8a377344539f5952bbe1aa10a7 P f9977432298448988bdb259803c1171b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:22:58.482358 31903 raft_consensus.cc:3037] T a4579a8a377344539f5952bbe1aa10a7 P f9977432298448988bdb259803c1171b [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:58.483358 31903 raft_consensus.cc:504] T a4579a8a377344539f5952bbe1aa10a7 P f9977432298448988bdb259803c1171b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c00472449774db482dac0a3c10272ed" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 38285 } } peers { permanent_uuid: "f9977432298448988bdb259803c1171b" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 33107 } } peers { permanent_uuid: "086e23e953ee4ecdb9a68942c3037fd5" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41097 } }
I20260808 18:22:58.483809 31903 leader_election.cc:288] T a4579a8a377344539f5952bbe1aa10a7 P f9977432298448988bdb259803c1171b [CANDIDATE]: Term 1 election: Requested vote from peers 0c00472449774db482dac0a3c10272ed (127.23.72.129:38285), 086e23e953ee4ecdb9a68942c3037fd5 (127.23.72.131:41097)
I20260808 18:22:58.484110 31504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4579a8a377344539f5952bbe1aa10a7" candidate_uuid: "f9977432298448988bdb259803c1171b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c00472449774db482dac0a3c10272ed"
I20260808 18:22:58.484184 31802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4579a8a377344539f5952bbe1aa10a7" candidate_uuid: "f9977432298448988bdb259803c1171b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "086e23e953ee4ecdb9a68942c3037fd5"
I20260808 18:22:58.484251 31504 raft_consensus.cc:3037] T a4579a8a377344539f5952bbe1aa10a7 P 0c00472449774db482dac0a3c10272ed [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:58.484350 31802 raft_consensus.cc:3037] T a4579a8a377344539f5952bbe1aa10a7 P 086e23e953ee4ecdb9a68942c3037fd5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:58.485246 31504 raft_consensus.cc:2454] T a4579a8a377344539f5952bbe1aa10a7 P 0c00472449774db482dac0a3c10272ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9977432298448988bdb259803c1171b in term 1.
I20260808 18:22:58.485462 31802 raft_consensus.cc:2454] T a4579a8a377344539f5952bbe1aa10a7 P 086e23e953ee4ecdb9a68942c3037fd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9977432298448988bdb259803c1171b in term 1.
I20260808 18:22:58.485517 31581 leader_election.cc:302] T a4579a8a377344539f5952bbe1aa10a7 P f9977432298448988bdb259803c1171b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c00472449774db482dac0a3c10272ed, f9977432298448988bdb259803c1171b; no voters:
I20260808 18:22:58.485687 31903 raft_consensus.cc:2781] T a4579a8a377344539f5952bbe1aa10a7 P f9977432298448988bdb259803c1171b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:22:58.485881 31903 raft_consensus.cc:686] T a4579a8a377344539f5952bbe1aa10a7 P f9977432298448988bdb259803c1171b [term 1 LEADER]: Becoming Leader. State: Replica: f9977432298448988bdb259803c1171b, State: Running, Role: LEADER
I20260808 18:22:58.486214 31903 consensus_queue.cc:237] T a4579a8a377344539f5952bbe1aa10a7 P f9977432298448988bdb259803c1171b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c00472449774db482dac0a3c10272ed" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 38285 } } peers { permanent_uuid: "f9977432298448988bdb259803c1171b" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 33107 } } peers { permanent_uuid: "086e23e953ee4ecdb9a68942c3037fd5" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41097 } }
I20260808 18:22:58.489532 30483 catalog_manager.cc:5495] T a4579a8a377344539f5952bbe1aa10a7 P f9977432298448988bdb259803c1171b reported cstate change: term changed from 0 to 1, leader changed from <none> to f9977432298448988bdb259803c1171b (127.23.72.130). New cstate: current_term: 1 leader_uuid: "f9977432298448988bdb259803c1171b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c00472449774db482dac0a3c10272ed" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 38285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9977432298448988bdb259803c1171b" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "086e23e953ee4ecdb9a68942c3037fd5" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41097 } health_report { overall_health: UNKNOWN } } }
W20260808 18:22:58.508101 31707 tablet.cc:2369] T a4579a8a377344539f5952bbe1aa10a7 P f9977432298448988bdb259803c1171b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:22:58.553117 31893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:58.549760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45414 (local address 127.23.72.130:33107)
0808 18:22:58.549902 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:58.549907 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:58.549924 (+ 17us) server_negotiation.cc:407] Connection header received
0808 18:22:58.550083 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:58.550086 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:58.550141 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:58.550252 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:22:58.550510 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.551009 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.551869 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.552075 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.552151 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:58.552402 (+ 251us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:22:58.552797 (+ 395us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:22:58.552889 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:58.552960 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 18:22:58.553010 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 18:22:58.558861 31802 raft_consensus.cc:1261] T a4579a8a377344539f5952bbe1aa10a7 P 086e23e953ee4ecdb9a68942c3037fd5 [term 1 FOLLOWER]: Refusing update from remote peer f9977432298448988bdb259803c1171b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:22:58.558856 31504 raft_consensus.cc:1261] T a4579a8a377344539f5952bbe1aa10a7 P 0c00472449774db482dac0a3c10272ed [term 1 FOLLOWER]: Refusing update from remote peer f9977432298448988bdb259803c1171b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:22:58.559620 31904 consensus_queue.cc:1035] T a4579a8a377344539f5952bbe1aa10a7 P f9977432298448988bdb259803c1171b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c00472449774db482dac0a3c10272ed" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 38285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:22:58.559855 31903 consensus_queue.cc:1035] T a4579a8a377344539f5952bbe1aa10a7 P f9977432298448988bdb259803c1171b [LEADER]: Connected to new peer: Peer: permanent_uuid: "086e23e953ee4ecdb9a68942c3037fd5" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:22:58.568923 31909 mvcc.cc:205] Tried to move back new op lower bound from 7316329998569889792 to 7316329998282240000. Current Snapshot: MvccSnapshot[applied={T|T < 7316329998569889792}]
I20260808 18:22:58.579006 30483 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:22:58.579151 30483 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:22:58.583602 30483 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e9d826ba0df445bfb8cc41e8dc8c9f18: Sending DeleteTablet for 3 replicas of tablet a4579a8a377344539f5952bbe1aa10a7
I20260808 18:22:58.584514 31782 tablet_service.cc:1514] Processing DeleteTablet for tablet a4579a8a377344539f5952bbe1aa10a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:22:58 UTC) from {username='oryx', principal='oryx/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:35196
I20260808 18:22:58.584789 31632 tablet_service.cc:1514] Processing DeleteTablet for tablet a4579a8a377344539f5952bbe1aa10a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:22:58 UTC) from {username='oryx', principal='oryx/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:45400
I20260808 18:22:58.585141 31918 tablet_replica.cc:330] T a4579a8a377344539f5952bbe1aa10a7 P 086e23e953ee4ecdb9a68942c3037fd5: stopping tablet replica
I20260808 18:22:58.585215 31919 tablet_replica.cc:330] T a4579a8a377344539f5952bbe1aa10a7 P f9977432298448988bdb259803c1171b: stopping tablet replica
I20260808 18:22:58.585520 31919 raft_consensus.cc:2229] T a4579a8a377344539f5952bbe1aa10a7 P f9977432298448988bdb259803c1171b [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:22:58.585532 31918 raft_consensus.cc:2229] T a4579a8a377344539f5952bbe1aa10a7 P 086e23e953ee4ecdb9a68942c3037fd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:22:58.585884 31919 raft_consensus.cc:2258] T a4579a8a377344539f5952bbe1aa10a7 P f9977432298448988bdb259803c1171b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:22:58.586006 31918 raft_consensus.cc:2258] T a4579a8a377344539f5952bbe1aa10a7 P 086e23e953ee4ecdb9a68942c3037fd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:22:58.587514 31919 ts_tablet_manager.cc:1905] T a4579a8a377344539f5952bbe1aa10a7 P f9977432298448988bdb259803c1171b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:22:58.587672 31918 ts_tablet_manager.cc:1905] T a4579a8a377344539f5952bbe1aa10a7 P 086e23e953ee4ecdb9a68942c3037fd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:22:58.587898 31481 tablet_service.cc:1514] Processing DeleteTablet for tablet a4579a8a377344539f5952bbe1aa10a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:22:58 UTC) from {username='oryx', principal='oryx/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:57772
I20260808 18:22:58.588923 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 30448
I20260808 18:22:58.592424 31919 ts_tablet_manager.cc:1918] T a4579a8a377344539f5952bbe1aa10a7 P f9977432298448988bdb259803c1171b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:22:58.592533 31919 log.cc:1196] T a4579a8a377344539f5952bbe1aa10a7 P f9977432298448988bdb259803c1171b: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-1/wal/wals/a4579a8a377344539f5952bbe1aa10a7
I20260808 18:22:58.592566 31918 ts_tablet_manager.cc:1918] T a4579a8a377344539f5952bbe1aa10a7 P 086e23e953ee4ecdb9a68942c3037fd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:22:58.592759 31918 log.cc:1196] T a4579a8a377344539f5952bbe1aa10a7 P 086e23e953ee4ecdb9a68942c3037fd5: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-2/wal/wals/a4579a8a377344539f5952bbe1aa10a7
I20260808 18:22:58.592880 31919 ts_tablet_manager.cc:1939] T a4579a8a377344539f5952bbe1aa10a7 P f9977432298448988bdb259803c1171b: Deleting consensus metadata
I20260808 18:22:58.593200 31918 ts_tablet_manager.cc:1939] T a4579a8a377344539f5952bbe1aa10a7 P 086e23e953ee4ecdb9a68942c3037fd5: Deleting consensus metadata
I20260808 18:22:58.596091 31921 tablet_replica.cc:330] T a4579a8a377344539f5952bbe1aa10a7 P 0c00472449774db482dac0a3c10272ed: stopping tablet replica
I20260808 18:22:58.596493 31921 raft_consensus.cc:2229] T a4579a8a377344539f5952bbe1aa10a7 P 0c00472449774db482dac0a3c10272ed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:22:58.596874 31921 raft_consensus.cc:2258] T a4579a8a377344539f5952bbe1aa10a7 P 0c00472449774db482dac0a3c10272ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:22:58.599254 31921 ts_tablet_manager.cc:1905] T a4579a8a377344539f5952bbe1aa10a7 P 0c00472449774db482dac0a3c10272ed: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 18:22:58.599555 31730 connection.cc:535] server connection from 127.0.0.1:35196 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 18:22:58.600037 31583 connection.cc:535] server connection from 127.0.0.1:45400 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 18:22:58.602177 31921 ts_tablet_manager.cc:1918] T a4579a8a377344539f5952bbe1aa10a7 P 0c00472449774db482dac0a3c10272ed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:22:58.602358 31921 log.cc:1196] T a4579a8a377344539f5952bbe1aa10a7 P 0c00472449774db482dac0a3c10272ed: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-0/wal/wals/a4579a8a377344539f5952bbe1aa10a7
I20260808 18:22:58.602753 31921 ts_tablet_manager.cc:1939] T a4579a8a377344539f5952bbe1aa10a7 P 0c00472449774db482dac0a3c10272ed: Deleting consensus metadata
W20260808 18:22:58.603968 31435 connection.cc:441] server connection from 127.0.0.1:57772 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57772 (request call id 1) could send its response
I20260808 18:22:58.608695 31870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:58.587336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40910 (local address 127.23.72.189:38137)
0808 18:22:58.587497 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:58.587502 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:58.595747 (+ 8245us) server_negotiation.cc:407] Connection header received
0808 18:22:58.595965 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:58.595971 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:58.596044 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:58.596134 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:58.597290 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.598129 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.600878 (+ 2749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.601255 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.601340 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:58.603330 (+ 1990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:58.603335 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:58.603338 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:58.603365 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:58.605246 (+ 1881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:58.605767 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:58.605771 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:58.605773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:58.605857 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:58.606142 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:58.606145 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:58.606147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:58.606358 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:58.606481 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:58.608413 (+ 1932us) server_negotiation.cc:299] Negotiation successful
0808 18:22:58.608538 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 18:22:58.613444 31560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:58.603263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45074 (local address 127.23.72.188:37769)
0808 18:22:58.603387 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:58.603391 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:58.604003 (+ 612us) server_negotiation.cc:407] Connection header received
0808 18:22:58.604205 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:58.604212 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:58.604277 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:58.604336 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:58.605247 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.606070 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.607249 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.607615 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.607692 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:58.609688 (+ 1996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:58.609695 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:58.609700 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:58.609730 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:58.611442 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:58.611961 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:58.611965 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:58.611968 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:58.612063 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:58.612414 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:58.612420 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:58.612422 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:58.612640 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:58.612750 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:58.613139 (+ 389us) server_negotiation.cc:299] Negotiation successful
0808 18:22:58.613256 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 18:22:58.614588 31288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:58.604608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40926 (local address 127.23.72.189:38137)
0808 18:22:58.604730 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:58.604736 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:58.605271 (+ 535us) server_negotiation.cc:407] Connection header received
0808 18:22:58.605427 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:58.605430 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:58.605471 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:58.605519 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:58.606331 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.607106 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.608249 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.608476 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.608589 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:58.610980 (+ 2391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:58.610984 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:58.610988 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:58.611011 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:58.612684 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:58.613183 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:58.613186 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:58.613188 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:58.613266 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:58.613611 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:58.613617 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:58.613620 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:58.613856 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:58.613977 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:58.614321 (+ 344us) server_negotiation.cc:299] Negotiation successful
0808 18:22:58.614431 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 18:22:58.617194 31874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:58.588299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45060 (local address 127.23.72.188:37769)
0808 18:22:58.588428 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:58.588432 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:58.607255 (+ 18823us) server_negotiation.cc:407] Connection header received
0808 18:22:58.607299 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:58.607302 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:58.607363 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:58.607462 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:58.609257 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.610479 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.611605 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:58.611942 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:58.612036 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:58.613768 (+ 1732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:58.613772 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:58.613775 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:58.613799 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:58.615681 (+ 1882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:58.616064 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:58.616068 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:58.616070 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:58.616151 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:58.616394 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:58.616397 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:58.616399 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:58.616583 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:58.616699 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:58.616937 (+ 238us) server_negotiation.cc:299] Negotiation successful
0808 18:22:58.617050 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 18:22:59.574004 31930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:59.573543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:36587 (local address 127.23.72.129:34409)
0808 18:22:59.573792 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:59.573878 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":96,"threads_started":1}
I20260808 18:22:59.574280 31929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:59.573081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:36587 (local address 127.23.72.131:52985)
0808 18:22:59.574072 (+ 991us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:59.574175 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":902,"thread_start_us":79,"threads_started":1}
W20260808 18:22:59.574589 31556 heartbeater.cc:644] Failed to heartbeat to 127.23.72.190:36587 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.23.72.190:36587: connect: Connection refused (error 111)
W20260808 18:22:59.574724 31858 heartbeater.cc:644] Failed to heartbeat to 127.23.72.190:36587 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.23.72.190:36587: connect: Connection refused (error 111)
I20260808 18:22:59.574808 31931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:59.574114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:36587 (local address 127.23.72.130:55045)
0808 18:22:59.574576 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:59.574677 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":57,"threads_started":1}
W20260808 18:22:59.575134 31705 heartbeater.cc:644] Failed to heartbeat to 127.23.72.190:36587 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.23.72.190:36587: connect: Connection refused (error 111)
I20260808 18:23:00.084196 31933 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9d826ba0df445bfb8cc41e8dc8c9f18)
I20260808 18:23:00.084226 31935 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9d826ba0df445bfb8cc41e8dc8c9f18)
I20260808 18:23:00.084437 31935 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } peers { permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 38137 } attrs { promote: false } } peers { permanent_uuid: "9d31cd09bc7843f4854913a5b460ba84" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 37769 } attrs { promote: false } }
I20260808 18:23:00.084438 31933 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } peers { permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 38137 } attrs { promote: false } } peers { permanent_uuid: "9d31cd09bc7843f4854913a5b460ba84" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 37769 } attrs { promote: false } }
I20260808 18:23:00.085500 31933 leader_election.cc:288] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9d826ba0df445bfb8cc41e8dc8c9f18 (127.23.72.190:36587), 11d877a7b3c344f89673448718d22e9a (127.23.72.189:38137)
I20260808 18:23:00.085651 31935 leader_election.cc:288] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9d826ba0df445bfb8cc41e8dc8c9f18 (127.23.72.190:36587), 9d31cd09bc7843f4854913a5b460ba84 (127.23.72.188:37769)
I20260808 18:23:00.086236 31936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:00.085624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:36587 (local address 127.0.0.1:49886)
0808 18:23:00.085996 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:00.086121 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":84,"threads_started":1}
W20260808 18:23:00.086506 31136 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.72.190:36587: connect: Connection refused (error 111)
I20260808 18:23:00.086802 31942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:00.086248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:36587 (local address 127.0.0.1:49902)
0808 18:23:00.086617 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:00.086689 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":71,"threads_started":1}
W20260808 18:23:00.086999 31342 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.72.190:36587: connect: Connection refused (error 111)
I20260808 18:23:00.087497 31936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:00.087196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:36587 (local address 127.0.0.1:49918)
0808 18:23:00.087335 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:00.087400 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260808 18:23:00.087598 31136 leader_election.cc:334] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e9d826ba0df445bfb8cc41e8dc8c9f18 (127.23.72.190:36587): Network error: Client connection negotiation failed: client connection to 127.23.72.190:36587: connect: Connection refused (error 111)
I20260808 18:23:00.087904 31942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:00.087667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:36587 (local address 127.0.0.1:49924)
0808 18:23:00.087797 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:00.087856 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260808 18:23:00.088052 31342 leader_election.cc:334] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e9d826ba0df445bfb8cc41e8dc8c9f18 (127.23.72.190:36587): Network error: Client connection negotiation failed: client connection to 127.23.72.190:36587: connect: Connection refused (error 111)
I20260808 18:23:00.090221 31937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:00.085716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.189:38137 (local address 127.0.0.1:40934)
0808 18:23:00.085983 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:00.086002 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:00.086122 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:00.086741 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:00.086746 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:00.086766 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:23:00.086891 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:00.086897 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:00.088129 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:00.088132 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:00.088992 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:00.089001 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:00.089725 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:00.089728 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:00.089901 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:00.089928 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 18:23:00.090007 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 18:23:00.090091 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":58,"thread_start_us":93,"threads_started":1}
I20260808 18:23:00.090227 31939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:00.085984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40934 (local address 127.23.72.189:38137)
0808 18:23:00.086471 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:00.086475 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:00.086489 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:23:00.086538 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:00.086542 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:00.086588 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:00.086995 (+ 407us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:23:00.087085 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:00.088008 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:00.089113 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:00.089621 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:00.089699 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:00.089745 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:00.090026 (+ 281us) server_negotiation.cc:299] Negotiation successful
0808 18:23:00.090114 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":66,"threads_started":1}
I20260808 18:23:00.090499 31938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:00.085624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:37769 (local address 127.0.0.1:45086)
0808 18:23:00.086041 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:00.086055 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:00.086153 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:00.086823 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:00.086826 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:00.086841 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:23:00.086951 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:00.086960 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:00.088384 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:00.088386 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:00.089308 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:00.089317 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:00.090104 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:00.090107 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:00.090191 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:00.090210 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 18:23:00.090287 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 18:23:00.090361 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"mutex_wait_us":183,"thread_start_us":59,"threads_started":1}
I20260808 18:23:00.090727 31940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:00.086267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45086 (local address 127.23.72.188:37769)
0808 18:23:00.086503 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:00.086509 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:00.086521 (+ 12us) server_negotiation.cc:407] Connection header received
0808 18:23:00.086642 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:00.086650 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:00.086706 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:00.087301 (+ 595us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:23:00.087397 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:00.088279 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:00.089431 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:00.090007 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:00.090094 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:00.090142 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:00.090443 (+ 301us) server_negotiation.cc:299] Negotiation successful
0808 18:23:00.090513 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":171,"thread_start_us":53,"threads_started":1}
I20260808 18:23:00.090865 31162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9d31cd09bc7843f4854913a5b460ba84" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "11d877a7b3c344f89673448718d22e9a" is_pre_election: true
I20260808 18:23:00.091079 31162 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d31cd09bc7843f4854913a5b460ba84 in term 1.
I20260808 18:23:00.091136 31368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "11d877a7b3c344f89673448718d22e9a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9d31cd09bc7843f4854913a5b460ba84" is_pre_election: true
I20260808 18:23:00.091337 31368 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11d877a7b3c344f89673448718d22e9a in term 1.
I20260808 18:23:00.091461 31344 leader_election.cc:302] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11d877a7b3c344f89673448718d22e9a, 9d31cd09bc7843f4854913a5b460ba84; no voters: e9d826ba0df445bfb8cc41e8dc8c9f18
I20260808 18:23:00.091673 31933 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 18:23:00.091708 31135 leader_election.cc:302] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11d877a7b3c344f89673448718d22e9a, 9d31cd09bc7843f4854913a5b460ba84; no voters: e9d826ba0df445bfb8cc41e8dc8c9f18
I20260808 18:23:00.091742 31933 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e9d826ba0df445bfb8cc41e8dc8c9f18)
I20260808 18:23:00.091789 31933 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [term 1 FOLLOWER]: Advancing to term 2
I20260808 18:23:00.091917 31935 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 18:23:00.091982 31935 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [term 1 FOLLOWER]: Starting leader election (detected failure of leader e9d826ba0df445bfb8cc41e8dc8c9f18)
I20260808 18:23:00.092043 31935 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [term 1 FOLLOWER]: Advancing to term 2
I20260808 18:23:00.093139 31933 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } peers { permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 38137 } attrs { promote: false } } peers { permanent_uuid: "9d31cd09bc7843f4854913a5b460ba84" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 37769 } attrs { promote: false } }
I20260808 18:23:00.093248 31935 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } peers { permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 38137 } attrs { promote: false } } peers { permanent_uuid: "9d31cd09bc7843f4854913a5b460ba84" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 37769 } attrs { promote: false } }
I20260808 18:23:00.093540 31933 leader_election.cc:288] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [CANDIDATE]: Term 2 election: Requested vote from peers e9d826ba0df445bfb8cc41e8dc8c9f18 (127.23.72.190:36587), 11d877a7b3c344f89673448718d22e9a (127.23.72.189:38137)
I20260808 18:23:00.093575 31935 leader_election.cc:288] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [CANDIDATE]: Term 2 election: Requested vote from peers e9d826ba0df445bfb8cc41e8dc8c9f18 (127.23.72.190:36587), 9d31cd09bc7843f4854913a5b460ba84 (127.23.72.188:37769)
I20260808 18:23:00.093873 31937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:00.093648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:36587 (local address 127.0.0.1:49936)
0808 18:23:00.093761 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:00.093814 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 18:23:00.093940 31368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "11d877a7b3c344f89673448718d22e9a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9d31cd09bc7843f4854913a5b460ba84"
I20260808 18:23:00.094029 31938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:00.093648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:36587 (local address 127.0.0.1:49938)
0808 18:23:00.093857 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:00.093911 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146}
I20260808 18:23:00.094086 31368 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 11d877a7b3c344f89673448718d22e9a in current term 2: Already voted for candidate 9d31cd09bc7843f4854913a5b460ba84 in this term.
I20260808 18:23:00.094084 31162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9d31cd09bc7843f4854913a5b460ba84" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "11d877a7b3c344f89673448718d22e9a"
I20260808 18:23:00.094209 31162 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9d31cd09bc7843f4854913a5b460ba84 in current term 2: Already voted for candidate 11d877a7b3c344f89673448718d22e9a in this term.
I20260808 18:23:00.094625 31938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:00.094403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:36587 (local address 127.0.0.1:49956)
0808 18:23:00.094523 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:00.094570 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260808 18:23:00.094637 31937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:00.094396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:36587 (local address 127.0.0.1:49948)
0808 18:23:00.094547 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:00.094587 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
W20260808 18:23:00.094751 31136 leader_election.cc:334] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e9d826ba0df445bfb8cc41e8dc8c9f18 (127.23.72.190:36587): Network error: Client connection negotiation failed: client connection to 127.23.72.190:36587: connect: Connection refused (error 111)
I20260808 18:23:00.094828 31136 leader_election.cc:302] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11d877a7b3c344f89673448718d22e9a; no voters: 9d31cd09bc7843f4854913a5b460ba84, e9d826ba0df445bfb8cc41e8dc8c9f18
I20260808 18:23:00.095000 31935 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260808 18:23:00.095106 31342 leader_election.cc:334] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e9d826ba0df445bfb8cc41e8dc8c9f18 (127.23.72.190:36587): Network error: Client connection negotiation failed: client connection to 127.23.72.190:36587: connect: Connection refused (error 111)
I20260808 18:23:00.095176 31342 leader_election.cc:302] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9d31cd09bc7843f4854913a5b460ba84; no voters: 11d877a7b3c344f89673448718d22e9a, e9d826ba0df445bfb8cc41e8dc8c9f18
I20260808 18:23:00.095295 31933 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260808 18:23:00.575748 31946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:00.575198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:36587 (local address 127.23.72.129:40303)
0808 18:23:00.575577 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:00.575643 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":101,"threads_started":1}
I20260808 18:23:00.575748 31945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:00.575198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:36587 (local address 127.23.72.131:33403)
0808 18:23:00.575577 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:00.575643 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":99,"threads_started":1}
I20260808 18:23:00.576411 31947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:00.575693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:36587 (local address 127.23.72.130:35365)
0808 18:23:00.576250 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:00.576298 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":84,"threads_started":1}
I20260808 18:23:01.577067 31949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:01.576579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:36587 (local address 127.23.72.131:43819)
0808 18:23:01.576873 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:01.576946 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":71,"threads_started":1}
I20260808 18:23:01.577500 31950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:01.577010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:36587 (local address 127.23.72.130:47139)
0808 18:23:01.577325 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:01.577386 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":60,"threads_started":1}
I20260808 18:23:01.577833 31948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:01.576534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:36587 (local address 127.23.72.129:54473)
0808 18:23:01.577680 (+ 1146us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:01.577741 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":96,"threads_started":1}
I20260808 18:23:02.046657 31951 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:23:02.046794 31951 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } peers { permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 38137 } attrs { promote: false } } peers { permanent_uuid: "9d31cd09bc7843f4854913a5b460ba84" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 37769 } attrs { promote: false } }
I20260808 18:23:02.047207 31951 leader_election.cc:288] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9d826ba0df445bfb8cc41e8dc8c9f18 (127.23.72.190:36587), 11d877a7b3c344f89673448718d22e9a (127.23.72.189:38137)
I20260808 18:23:02.047755 31162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9d31cd09bc7843f4854913a5b460ba84" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "11d877a7b3c344f89673448718d22e9a" is_pre_election: true
I20260808 18:23:02.047897 31162 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d31cd09bc7843f4854913a5b460ba84 in term 2.
I20260808 18:23:02.047955 31952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:02.047505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:36587 (local address 127.0.0.1:50254)
0808 18:23:02.047751 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:02.047815 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":97,"threads_started":1}
I20260808 18:23:02.048318 31344 leader_election.cc:302] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11d877a7b3c344f89673448718d22e9a, 9d31cd09bc7843f4854913a5b460ba84; no voters:
I20260808 18:23:02.048514 31951 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260808 18:23:02.048591 31951 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:23:02.048635 31951 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [term 2 FOLLOWER]: Advancing to term 3
I20260808 18:23:02.048928 31952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:02.048624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:36587 (local address 127.0.0.1:50260)
0808 18:23:02.048788 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:02.048864 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260808 18:23:02.049081 31342 leader_election.cc:334] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e9d826ba0df445bfb8cc41e8dc8c9f18 (127.23.72.190:36587): Network error: Client connection negotiation failed: client connection to 127.23.72.190:36587: connect: Connection refused (error 111)
I20260808 18:23:02.049880 31951 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } peers { permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 38137 } attrs { promote: false } } peers { permanent_uuid: "9d31cd09bc7843f4854913a5b460ba84" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 37769 } attrs { promote: false } }
I20260808 18:23:02.050230 31951 leader_election.cc:288] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [CANDIDATE]: Term 3 election: Requested vote from peers e9d826ba0df445bfb8cc41e8dc8c9f18 (127.23.72.190:36587), 11d877a7b3c344f89673448718d22e9a (127.23.72.189:38137)
I20260808 18:23:02.050534 31952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:02.050321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:36587 (local address 127.0.0.1:50274)
0808 18:23:02.050406 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:02.050460 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260808 18:23:02.050720 31162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9d31cd09bc7843f4854913a5b460ba84" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "11d877a7b3c344f89673448718d22e9a"
I20260808 18:23:02.050961 31162 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [term 2 FOLLOWER]: Advancing to term 3
I20260808 18:23:02.051198 31952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:02.050991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:36587 (local address 127.0.0.1:50290)
0808 18:23:02.051095 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:02.051149 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 18:23:02.051297 31342 leader_election.cc:334] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer e9d826ba0df445bfb8cc41e8dc8c9f18 (127.23.72.190:36587): Network error: Client connection negotiation failed: client connection to 127.23.72.190:36587: connect: Connection refused (error 111)
I20260808 18:23:02.052232 31162 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d31cd09bc7843f4854913a5b460ba84 in term 3.
I20260808 18:23:02.052597 31344 leader_election.cc:302] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11d877a7b3c344f89673448718d22e9a, 9d31cd09bc7843f4854913a5b460ba84; no voters: e9d826ba0df445bfb8cc41e8dc8c9f18
I20260808 18:23:02.052896 31951 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 18:23:02.053141 31951 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [term 3 LEADER]: Becoming Leader. State: Replica: 9d31cd09bc7843f4854913a5b460ba84, State: Running, Role: LEADER
I20260808 18:23:02.053442 31951 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } peers { permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 38137 } attrs { promote: false } } peers { permanent_uuid: "9d31cd09bc7843f4854913a5b460ba84" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 37769 } attrs { promote: false } }
I20260808 18:23:02.054881 31956 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d31cd09bc7843f4854913a5b460ba84. Latest consensus state: current_term: 3 leader_uuid: "9d31cd09bc7843f4854913a5b460ba84" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } peers { permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 38137 } attrs { promote: false } } peers { permanent_uuid: "9d31cd09bc7843f4854913a5b460ba84" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 37769 } attrs { promote: false } } }
I20260808 18:23:02.055001 31956 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [sys.catalog]: This master's current role is: LEADER
I20260808 18:23:02.055279 31958 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:23:02.057483 31958 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:23:02.057874 31958 catalog_manager.cc:1259] Loaded cluster ID: bf3f1e45a3cf47969ef2d1762c680e0e
I20260808 18:23:02.057927 31958 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:23:02.058593 31958 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:23:02.059000 31958 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84: Loaded TSK: 0
I20260808 18:23:02.059803 31958 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:23:02.076947 31358 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45074:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:23:02.080286 31358 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:23:02.083320 31952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:02.083050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:36587 (local address 127.0.0.1:50306)
0808 18:23:02.083174 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:02.083238 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 18:23:02.083498 31162 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [term 3 FOLLOWER]: Refusing update from remote peer 9d31cd09bc7843f4854913a5b460ba84: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260808 18:23:02.084044 31952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:02.083804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:36587 (local address 127.0.0.1:50320)
0808 18:23:02.083925 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:02.083980 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 18:23:02.084121 31951 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 38137 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260808 18:23:02.084340 31342 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 -> Peer e9d826ba0df445bfb8cc41e8dc8c9f18 (127.23.72.190:36587): Couldn't send request to peer e9d826ba0df445bfb8cc41e8dc8c9f18. Status: Network error: Client connection negotiation failed: client connection to 127.23.72.190:36587: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 18:23:02.085721 31961 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d31cd09bc7843f4854913a5b460ba84. Latest consensus state: current_term: 3 leader_uuid: "9d31cd09bc7843f4854913a5b460ba84" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } peers { permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 38137 } attrs { promote: false } } peers { permanent_uuid: "9d31cd09bc7843f4854913a5b460ba84" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 37769 } attrs { promote: false } } }
I20260808 18:23:02.085867 31961 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:23:02.087023 31956 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "9d31cd09bc7843f4854913a5b460ba84" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } peers { permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 38137 } attrs { promote: false } } peers { permanent_uuid: "9d31cd09bc7843f4854913a5b460ba84" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 37769 } attrs { promote: false } } }
I20260808 18:23:02.087178 31956 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84 [sys.catalog]: This master's current role is: LEADER
I20260808 18:23:02.087311 31959 mvcc.cc:205] Tried to move back new op lower bound from 7316330013008756736 to 7316330012893429760. Current Snapshot: MvccSnapshot[applied={T|T < 7316330013008756736}]
I20260808 18:23:02.087489 31961 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "9d31cd09bc7843f4854913a5b460ba84" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e9d826ba0df445bfb8cc41e8dc8c9f18" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 36587 } } peers { permanent_uuid: "11d877a7b3c344f89673448718d22e9a" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 38137 } attrs { promote: false } } peers { permanent_uuid: "9d31cd09bc7843f4854913a5b460ba84" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 37769 } attrs { promote: false } } }
I20260808 18:23:02.087607 31961 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11d877a7b3c344f89673448718d22e9a [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:23:02.097675 31967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:02.093118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35200 (local address 127.23.72.131:41097)
0808 18:23:02.093426 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:02.093430 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:02.093464 (+ 34us) server_negotiation.cc:407] Connection header received
0808 18:23:02.093521 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:02.093527 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:02.093821 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:02.093916 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:23:02.094211 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:02.094983 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:02.096468 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:02.097114 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:02.097224 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:02.097288 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:02.097465 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 18:23:02.097536 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":142,"threads_started":1}
I20260808 18:23:02.097664 31952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:02.092968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.131:41097 (local address 127.0.0.1:35200)
0808 18:23:02.093090 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:02.093106 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:02.093193 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:02.093951 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:02.093954 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:02.093971 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:23:02.094076 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:02.094082 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:02.095093 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:02.095097 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:02.096330 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:02.096340 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:02.097241 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:02.097244 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:02.097332 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:02.097354 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 18:23:02.097447 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 18:23:02.097528 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":4}
I20260808 18:23:02.098088 31970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:02.093506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.130:33107 (local address 127.0.0.1:45420)
0808 18:23:02.094171 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:02.094186 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:02.094279 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:02.094604 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:02.094607 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:02.094623 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:23:02.094719 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:02.094724 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:02.095677 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:02.095682 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:02.096670 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:02.096678 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:02.097665 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:02.097669 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:02.097763 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:02.097784 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 18:23:02.097849 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 18:23:02.097929 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"mutex_wait_us":26,"thread_start_us":88,"threads_started":1}
I20260808 18:23:02.098290 31971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:02.094137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57782 (local address 127.23.72.129:38285)
0808 18:23:02.094386 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:02.094390 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:02.094416 (+ 26us) server_negotiation.cc:407] Connection header received
0808 18:23:02.094468 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:02.094470 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:02.094515 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:02.094593 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:23:02.094911 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:02.095608 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:02.097042 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:02.097655 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:02.097746 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:02.097802 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:02.098067 (+ 265us) server_negotiation.cc:299] Negotiation successful
0808 18:23:02.098155 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":73,"threads_started":1}
I20260808 18:23:02.098510 31969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:02.093615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45420 (local address 127.23.72.130:33107)
0808 18:23:02.093832 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:02.093836 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:02.094369 (+ 533us) server_negotiation.cc:407] Connection header received
0808 18:23:02.094431 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:02.094434 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:02.094494 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:02.094586 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:23:02.094827 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:02.095559 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:02.096809 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:02.097444 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:02.097544 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:02.097604 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:02.098300 (+ 696us) server_negotiation.cc:299] Negotiation successful
0808 18:23:02.098384 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":78,"threads_started":1}
I20260808 18:23:02.098516 31782 tablet_service.cc:1467] Processing CreateTablet for tablet 5cf7e5d1f2534894b2d005bb82618dba (DEFAULT_TABLE table=test-table [id=0fd6aa66f585402d86419475432a4589]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:23:02.098812 31782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5cf7e5d1f2534894b2d005bb82618dba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:23:02.099022 31632 tablet_service.cc:1467] Processing CreateTablet for tablet 5cf7e5d1f2534894b2d005bb82618dba (DEFAULT_TABLE table=test-table [id=0fd6aa66f585402d86419475432a4589]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:23:02.099303 31632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5cf7e5d1f2534894b2d005bb82618dba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:23:02.099469 31968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:02.093799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:38285 (local address 127.0.0.1:57782)
0808 18:23:02.094274 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:02.094286 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:02.094354 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:02.094696 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:02.094699 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:02.094715 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:23:02.094796 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:02.094801 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:02.095741 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:02.095745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:02.096922 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:02.096931 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:02.097858 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:02.097862 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:02.097936 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:02.097955 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 18:23:02.098027 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 18:23:02.098083 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"mutex_wait_us":23,"thread_start_us":83,"threads_started":1}
I20260808 18:23:02.100111 31481 tablet_service.cc:1467] Processing CreateTablet for tablet 5cf7e5d1f2534894b2d005bb82618dba (DEFAULT_TABLE table=test-table [id=0fd6aa66f585402d86419475432a4589]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:23:02.100383 31481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5cf7e5d1f2534894b2d005bb82618dba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:23:02.101681 31972 tablet_bootstrap.cc:492] T 5cf7e5d1f2534894b2d005bb82618dba P 086e23e953ee4ecdb9a68942c3037fd5: Bootstrap starting.
I20260808 18:23:02.101760 31973 tablet_bootstrap.cc:492] T 5cf7e5d1f2534894b2d005bb82618dba P f9977432298448988bdb259803c1171b: Bootstrap starting.
I20260808 18:23:02.102635 31972 tablet_bootstrap.cc:654] T 5cf7e5d1f2534894b2d005bb82618dba P 086e23e953ee4ecdb9a68942c3037fd5: Neither blocks nor log segments found. Creating new log.
I20260808 18:23:02.102650 31973 tablet_bootstrap.cc:654] T 5cf7e5d1f2534894b2d005bb82618dba P f9977432298448988bdb259803c1171b: Neither blocks nor log segments found. Creating new log.
I20260808 18:23:02.102916 31974 tablet_bootstrap.cc:492] T 5cf7e5d1f2534894b2d005bb82618dba P 0c00472449774db482dac0a3c10272ed: Bootstrap starting.
I20260808 18:23:02.103869 31972 tablet_bootstrap.cc:492] T 5cf7e5d1f2534894b2d005bb82618dba P 086e23e953ee4ecdb9a68942c3037fd5: No bootstrap required, opened a new log
I20260808 18:23:02.103967 31972 ts_tablet_manager.cc:1397] T 5cf7e5d1f2534894b2d005bb82618dba P 086e23e953ee4ecdb9a68942c3037fd5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 18:23:02.104079 31974 tablet_bootstrap.cc:654] T 5cf7e5d1f2534894b2d005bb82618dba P 0c00472449774db482dac0a3c10272ed: Neither blocks nor log segments found. Creating new log.
I20260808 18:23:02.104382 31972 raft_consensus.cc:348] T 5cf7e5d1f2534894b2d005bb82618dba P 086e23e953ee4ecdb9a68942c3037fd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "086e23e953ee4ecdb9a68942c3037fd5" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41097 } } peers { permanent_uuid: "f9977432298448988bdb259803c1171b" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 33107 } } peers { permanent_uuid: "0c00472449774db482dac0a3c10272ed" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 38285 } }
I20260808 18:23:02.104507 31972 raft_consensus.cc:374] T 5cf7e5d1f2534894b2d005bb82618dba P 086e23e953ee4ecdb9a68942c3037fd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:23:02.104552 31972 raft_consensus.cc:729] T 5cf7e5d1f2534894b2d005bb82618dba P 086e23e953ee4ecdb9a68942c3037fd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 086e23e953ee4ecdb9a68942c3037fd5, State: Initialized, Role: FOLLOWER
I20260808 18:23:02.104681 31972 consensus_queue.cc:260] T 5cf7e5d1f2534894b2d005bb82618dba P 086e23e953ee4ecdb9a68942c3037fd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "086e23e953ee4ecdb9a68942c3037fd5" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41097 } } peers { permanent_uuid: "f9977432298448988bdb259803c1171b" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 33107 } } peers { permanent_uuid: "0c00472449774db482dac0a3c10272ed" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 38285 } }
I20260808 18:23:02.105062 31972 ts_tablet_manager.cc:1428] T 5cf7e5d1f2534894b2d005bb82618dba P 086e23e953ee4ecdb9a68942c3037fd5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:23:02.105316 31974 tablet_bootstrap.cc:492] T 5cf7e5d1f2534894b2d005bb82618dba P 0c00472449774db482dac0a3c10272ed: No bootstrap required, opened a new log
I20260808 18:23:02.105396 31974 ts_tablet_manager.cc:1397] T 5cf7e5d1f2534894b2d005bb82618dba P 0c00472449774db482dac0a3c10272ed: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 18:23:02.105768 31980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:02.105368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:36587 (local address 127.23.72.131:58397)
0808 18:23:02.105599 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:02.105660 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":96,"threads_started":1}
I20260808 18:23:02.105921 31974 raft_consensus.cc:348] T 5cf7e5d1f2534894b2d005bb82618dba P 0c00472449774db482dac0a3c10272ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "086e23e953ee4ecdb9a68942c3037fd5" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41097 } } peers { permanent_uuid: "f9977432298448988bdb259803c1171b" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 33107 } } peers { permanent_uuid: "0c00472449774db482dac0a3c10272ed" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 38285 } }
I20260808 18:23:02.106062 31974 raft_consensus.cc:374] T 5cf7e5d1f2534894b2d005bb82618dba P 0c00472449774db482dac0a3c10272ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:23:02.106103 31974 raft_consensus.cc:729] T 5cf7e5d1f2534894b2d005bb82618dba P 0c00472449774db482dac0a3c10272ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c00472449774db482dac0a3c10272ed, State: Initialized, Role: FOLLOWER
I20260808 18:23:02.106235 31974 consensus_queue.cc:260] T 5cf7e5d1f2534894b2d005bb82618dba P 0c00472449774db482dac0a3c10272ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "086e23e953ee4ecdb9a68942c3037fd5" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41097 } } peers { permanent_uuid: "f9977432298448988bdb259803c1171b" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 33107 } } peers { permanent_uuid: "0c00472449774db482dac0a3c10272ed" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 38285 } }
I20260808 18:23:02.106593 31974 ts_tablet_manager.cc:1428] T 5cf7e5d1f2534894b2d005bb82618dba P 0c00472449774db482dac0a3c10272ed: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260808 18:23:02.108011 31980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:02.107736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:36587 (local address 127.23.72.131:51325)
0808 18:23:02.107852 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:02.107920 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 18:23:02.108247 31982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:02.107386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:36587 (local address 127.23.72.129:47127)
0808 18:23:02.108074 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:02.108130 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":605,"thread_start_us":147,"threads_started":1}
I20260808 18:23:02.108877 31973 tablet_bootstrap.cc:492] T 5cf7e5d1f2534894b2d005bb82618dba P f9977432298448988bdb259803c1171b: No bootstrap required, opened a new log
I20260808 18:23:02.108979 31973 ts_tablet_manager.cc:1397] T 5cf7e5d1f2534894b2d005bb82618dba P f9977432298448988bdb259803c1171b: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260808 18:23:02.109339 31973 raft_consensus.cc:348] T 5cf7e5d1f2534894b2d005bb82618dba P f9977432298448988bdb259803c1171b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "086e23e953ee4ecdb9a68942c3037fd5" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41097 } } peers { permanent_uuid: "f9977432298448988bdb259803c1171b" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 33107 } } peers { permanent_uuid: "0c00472449774db482dac0a3c10272ed" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 38285 } }
I20260808 18:23:02.109464 31973 raft_consensus.cc:374] T 5cf7e5d1f2534894b2d005bb82618dba P f9977432298448988bdb259803c1171b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:23:02.109510 31973 raft_consensus.cc:729] T 5cf7e5d1f2534894b2d005bb82618dba P f9977432298448988bdb259803c1171b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9977432298448988bdb259803c1171b, State: Initialized, Role: FOLLOWER
I20260808 18:23:02.109637 31973 consensus_queue.cc:260] T 5cf7e5d1f2534894b2d005bb82618dba P f9977432298448988bdb259803c1171b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "086e23e953ee4ecdb9a68942c3037fd5" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41097 } } peers { permanent_uuid: "f9977432298448988bdb259803c1171b" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 33107 } } peers { permanent_uuid: "0c00472449774db482dac0a3c10272ed" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 38285 } }
I20260808 18:23:02.109977 31973 ts_tablet_manager.cc:1428] T 5cf7e5d1f2534894b2d005bb82618dba P f9977432298448988bdb259803c1171b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:23:02.110625 31984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:02.110232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:36587 (local address 127.23.72.130:45943)
0808 18:23:02.110481 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:02.110539 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":105,"threads_started":1}
I20260808 18:23:02.111290 31704 heartbeater.cc:503] Master 127.23.72.188:37769 was elected leader, sending a full tablet report...
I20260808 18:23:02.111673 31984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:02.111174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:36587 (local address 127.23.72.130:51703)
0808 18:23:02.111538 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:02.111605 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":289}
I20260808 18:23:02.371390 31981 raft_consensus.cc:482] T 5cf7e5d1f2534894b2d005bb82618dba P 0c00472449774db482dac0a3c10272ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:23:02.371559 31981 raft_consensus.cc:504] T 5cf7e5d1f2534894b2d005bb82618dba P 0c00472449774db482dac0a3c10272ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "086e23e953ee4ecdb9a68942c3037fd5" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41097 } } peers { permanent_uuid: "f9977432298448988bdb259803c1171b" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 33107 } } peers { permanent_uuid: "0c00472449774db482dac0a3c10272ed" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 38285 } }
I20260808 18:23:02.372594 31981 leader_election.cc:288] T 5cf7e5d1f2534894b2d005bb82618dba P 0c00472449774db482dac0a3c10272ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 086e23e953ee4ecdb9a68942c3037fd5 (127.23.72.131:41097), f9977432298448988bdb259803c1171b (127.23.72.130:33107)
I20260808 18:23:02.376420 31967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:02.372948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:44353 (local address 127.23.72.131:41097)
0808 18:23:02.373117 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:02.373122 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:02.373138 (+ 16us) server_negotiation.cc:407] Connection header received
0808 18:23:02.373257 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:02.373260 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:02.373308 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:02.373384 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:23:02.373672 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:02.374192 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:02.375392 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:02.375832 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:02.375972 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:02.376025 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:02.376232 (+ 207us) server_negotiation.cc:299] Negotiation successful
0808 18:23:02.376320 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260808 18:23:02.376413 31982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:02.372817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.131:41097 (local address 127.23.72.129:44353)
0808 18:23:02.373018 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:02.373035 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:02.373138 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:02.373408 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:02.373411 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:02.373428 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:23:02.373560 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:02.373566 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:02.374314 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:02.374319 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:02.375268 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:02.375277 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:02.375969 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:02.375973 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:02.376081 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:02.376110 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 18:23:02.376204 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 18:23:02.376279 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":26}
I20260808 18:23:02.376502 31969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:02.372952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:47273 (local address 127.23.72.130:33107)
0808 18:23:02.373112 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:02.373117 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:02.373343 (+ 226us) server_negotiation.cc:407] Connection header received
0808 18:23:02.373501 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:02.373504 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:02.373549 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:02.373630 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:23:02.373910 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:02.374419 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:02.375494 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:02.375884 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:02.376023 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:02.376075 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:02.376300 (+ 225us) server_negotiation.cc:299] Negotiation successful
0808 18:23:02.376374 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 18:23:02.376561 31985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:02.372832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.130:33107 (local address 127.23.72.129:47273)
0808 18:23:02.373252 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:02.373263 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:02.373385 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:02.373641 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:02.373644 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:02.373659 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:23:02.373786 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:02.373792 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:02.374540 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:02.374545 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:02.375330 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:02.375339 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:02.376010 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:02.376014 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:02.376176 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:02.376194 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 18:23:02.376284 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 18:23:02.376350 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"mutex_wait_us":17,"thread_start_us":166,"threads_started":1}
I20260808 18:23:02.376917 31802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cf7e5d1f2534894b2d005bb82618dba" candidate_uuid: "0c00472449774db482dac0a3c10272ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "086e23e953ee4ecdb9a68942c3037fd5" is_pre_election: true
I20260808 18:23:02.377074 31802 raft_consensus.cc:2454] T 5cf7e5d1f2534894b2d005bb82618dba P 086e23e953ee4ecdb9a68942c3037fd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c00472449774db482dac0a3c10272ed in term 0.
I20260808 18:23:02.377087 31652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cf7e5d1f2534894b2d005bb82618dba" candidate_uuid: "0c00472449774db482dac0a3c10272ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9977432298448988bdb259803c1171b" is_pre_election: true
I20260808 18:23:02.377323 31652 raft_consensus.cc:2454] T 5cf7e5d1f2534894b2d005bb82618dba P f9977432298448988bdb259803c1171b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c00472449774db482dac0a3c10272ed in term 0.
I20260808 18:23:02.377463 31435 leader_election.cc:302] T 5cf7e5d1f2534894b2d005bb82618dba P 0c00472449774db482dac0a3c10272ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 086e23e953ee4ecdb9a68942c3037fd5, 0c00472449774db482dac0a3c10272ed; no voters:
I20260808 18:23:02.377765 31981 raft_consensus.cc:2781] T 5cf7e5d1f2534894b2d005bb82618dba P 0c00472449774db482dac0a3c10272ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:23:02.377831 31981 raft_consensus.cc:482] T 5cf7e5d1f2534894b2d005bb82618dba P 0c00472449774db482dac0a3c10272ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:23:02.377869 31981 raft_consensus.cc:3037] T 5cf7e5d1f2534894b2d005bb82618dba P 0c00472449774db482dac0a3c10272ed [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:23:02.378890 31981 raft_consensus.cc:504] T 5cf7e5d1f2534894b2d005bb82618dba P 0c00472449774db482dac0a3c10272ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "086e23e953ee4ecdb9a68942c3037fd5" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41097 } } peers { permanent_uuid: "f9977432298448988bdb259803c1171b" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 33107 } } peers { permanent_uuid: "0c00472449774db482dac0a3c10272ed" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 38285 } }
I20260808 18:23:02.379223 31981 leader_election.cc:288] T 5cf7e5d1f2534894b2d005bb82618dba P 0c00472449774db482dac0a3c10272ed [CANDIDATE]: Term 1 election: Requested vote from peers 086e23e953ee4ecdb9a68942c3037fd5 (127.23.72.131:41097), f9977432298448988bdb259803c1171b (127.23.72.130:33107)
I20260808 18:23:02.379522 31802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cf7e5d1f2534894b2d005bb82618dba" candidate_uuid: "0c00472449774db482dac0a3c10272ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "086e23e953ee4ecdb9a68942c3037fd5"
I20260808 18:23:02.379573 31652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cf7e5d1f2534894b2d005bb82618dba" candidate_uuid: "0c00472449774db482dac0a3c10272ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9977432298448988bdb259803c1171b"
I20260808 18:23:02.379631 31802 raft_consensus.cc:3037] T 5cf7e5d1f2534894b2d005bb82618dba P 086e23e953ee4ecdb9a68942c3037fd5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:23:02.379715 31652 raft_consensus.cc:3037] T 5cf7e5d1f2534894b2d005bb82618dba P f9977432298448988bdb259803c1171b [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:23:02.380893 31802 raft_consensus.cc:2454] T 5cf7e5d1f2534894b2d005bb82618dba P 086e23e953ee4ecdb9a68942c3037fd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c00472449774db482dac0a3c10272ed in term 1.
I20260808 18:23:02.381065 31652 raft_consensus.cc:2454] T 5cf7e5d1f2534894b2d005bb82618dba P f9977432298448988bdb259803c1171b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c00472449774db482dac0a3c10272ed in term 1.
I20260808 18:23:02.381234 31435 leader_election.cc:302] T 5cf7e5d1f2534894b2d005bb82618dba P 0c00472449774db482dac0a3c10272ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 086e23e953ee4ecdb9a68942c3037fd5, 0c00472449774db482dac0a3c10272ed; no voters:
I20260808 18:23:02.381424 31981 raft_consensus.cc:2781] T 5cf7e5d1f2534894b2d005bb82618dba P 0c00472449774db482dac0a3c10272ed [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:23:02.381700 31981 raft_consensus.cc:686] T 5cf7e5d1f2534894b2d005bb82618dba P 0c00472449774db482dac0a3c10272ed [term 1 LEADER]: Becoming Leader. State: Replica: 0c00472449774db482dac0a3c10272ed, State: Running, Role: LEADER
I20260808 18:23:02.381909 31555 heartbeater.cc:503] Master 127.23.72.188:37769 was elected leader, sending a full tablet report...
I20260808 18:23:02.381948 31981 consensus_queue.cc:237] T 5cf7e5d1f2534894b2d005bb82618dba P 0c00472449774db482dac0a3c10272ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "086e23e953ee4ecdb9a68942c3037fd5" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41097 } } peers { permanent_uuid: "f9977432298448988bdb259803c1171b" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 33107 } } peers { permanent_uuid: "0c00472449774db482dac0a3c10272ed" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 38285 } }
I20260808 18:23:02.382714 31985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:02.382370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:36587 (local address 127.23.72.129:57105)
0808 18:23:02.382570 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:02.382644 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
I20260808 18:23:02.384385 31358 catalog_manager.cc:5495] T 5cf7e5d1f2534894b2d005bb82618dba P 0c00472449774db482dac0a3c10272ed reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c00472449774db482dac0a3c10272ed (127.23.72.129). New cstate: current_term: 1 leader_uuid: "0c00472449774db482dac0a3c10272ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "086e23e953ee4ecdb9a68942c3037fd5" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9977432298448988bdb259803c1171b" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c00472449774db482dac0a3c10272ed" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 38285 } health_report { overall_health: HEALTHY } } }
I20260808 18:23:02.460516 31971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:02.456308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57790 (local address 127.23.72.129:38285)
0808 18:23:02.456452 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:02.456456 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:02.456470 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:23:02.456553 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:02.456556 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:02.456616 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:02.456716 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:23:02.456934 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:02.457504 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:02.458644 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:02.458942 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:02.459045 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:02.459462 (+ 417us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:23:02.460020 (+ 558us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:23:02.460112 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:02.460298 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 18:23:02.460370 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 18:23:02.461051 31968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:02.460759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:36587 (local address 127.0.0.1:50334)
0808 18:23:02.460907 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:02.460979 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260808 18:23:02.461740 31968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:02.461528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:36587 (local address 127.0.0.1:50340)
0808 18:23:02.461632 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:02.461687 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 18:23:02.463920 31652 raft_consensus.cc:1261] T 5cf7e5d1f2534894b2d005bb82618dba P f9977432298448988bdb259803c1171b [term 1 FOLLOWER]: Refusing update from remote peer 0c00472449774db482dac0a3c10272ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:23:02.464468 31802 raft_consensus.cc:1261] T 5cf7e5d1f2534894b2d005bb82618dba P 086e23e953ee4ecdb9a68942c3037fd5 [term 1 FOLLOWER]: Refusing update from remote peer 0c00472449774db482dac0a3c10272ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:23:02.464736 31986 consensus_queue.cc:1035] T 5cf7e5d1f2534894b2d005bb82618dba P 0c00472449774db482dac0a3c10272ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9977432298448988bdb259803c1171b" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:23:02.465016 31981 consensus_queue.cc:1035] T 5cf7e5d1f2534894b2d005bb82618dba P 0c00472449774db482dac0a3c10272ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "086e23e953ee4ecdb9a68942c3037fd5" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:23:02.466038 31857 heartbeater.cc:503] Master 127.23.72.188:37769 was elected leader, sending a full tablet report...
I20260808 18:23:02.466781 31980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:02.466456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:36587 (local address 127.23.72.131:53933)
0808 18:23:02.466652 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:02.466708 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
I20260808 18:23:02.467232 31984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:02.466949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:36587 (local address 127.23.72.130:59275)
0808 18:23:02.467107 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:02.467159 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260808 18:23:02.470472 31984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:02.470210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:36587 (local address 127.23.72.130:52489)
0808 18:23:02.470343 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:02.470394 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260808 18:23:02.470961 31993 mvcc.cc:205] Tried to move back new op lower bound from 7316330014565425152 to 7316330014239309824. Current Snapshot: MvccSnapshot[applied={T|T < 7316330014565425152}]
I20260808 18:23:02.471107 31985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:02.470823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:36587 (local address 127.23.72.129:50651)
0808 18:23:02.470964 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:02.471020 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260808 18:23:02.473053 31980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:02.472770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:36587 (local address 127.23.72.131:55049)
0808 18:23:02.472925 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:02.472983 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260808 18:23:02.479204 31357 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45074:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:23:02.479388 31357 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45074:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:23:02.483987 31357 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9d31cd09bc7843f4854913a5b460ba84: Sending DeleteTablet for 3 replicas of tablet 5cf7e5d1f2534894b2d005bb82618dba
I20260808 18:23:02.484705 31782 tablet_service.cc:1514] Processing DeleteTablet for tablet 5cf7e5d1f2534894b2d005bb82618dba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:23:02 UTC) from {username='oryx', principal='oryx/127.23.72.188@KRBTEST.COM'} at 127.0.0.1:35200
I20260808 18:23:02.484912 31481 tablet_service.cc:1514] Processing DeleteTablet for tablet 5cf7e5d1f2534894b2d005bb82618dba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:23:02 UTC) from {username='oryx', principal='oryx/127.23.72.188@KRBTEST.COM'} at 127.0.0.1:57782
I20260808 18:23:02.485147 31999 tablet_replica.cc:330] T 5cf7e5d1f2534894b2d005bb82618dba P 086e23e953ee4ecdb9a68942c3037fd5: stopping tablet replica
I20260808 18:23:02.485271 31999 raft_consensus.cc:2229] T 5cf7e5d1f2534894b2d005bb82618dba P 086e23e953ee4ecdb9a68942c3037fd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:23:02.485412 31999 raft_consensus.cc:2258] T 5cf7e5d1f2534894b2d005bb82618dba P 086e23e953ee4ecdb9a68942c3037fd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:23:02.485416 31632 tablet_service.cc:1514] Processing DeleteTablet for tablet 5cf7e5d1f2534894b2d005bb82618dba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:23:02 UTC) from {username='oryx', principal='oryx/127.23.72.188@KRBTEST.COM'} at 127.0.0.1:45420
I20260808 18:23:02.485849 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 31313
I20260808 18:23:02.486176 31999 ts_tablet_manager.cc:1905] T 5cf7e5d1f2534894b2d005bb82618dba P 086e23e953ee4ecdb9a68942c3037fd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:23:02.486676 32001 tablet_replica.cc:330] T 5cf7e5d1f2534894b2d005bb82618dba P f9977432298448988bdb259803c1171b: stopping tablet replica
I20260808 18:23:02.486896 32001 raft_consensus.cc:2229] T 5cf7e5d1f2534894b2d005bb82618dba P f9977432298448988bdb259803c1171b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:23:02.487064 32001 raft_consensus.cc:2258] T 5cf7e5d1f2534894b2d005bb82618dba P f9977432298448988bdb259803c1171b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:23:02.488117 32001 ts_tablet_manager.cc:1905] T 5cf7e5d1f2534894b2d005bb82618dba P f9977432298448988bdb259803c1171b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:23:02.489681 31999 ts_tablet_manager.cc:1918] T 5cf7e5d1f2534894b2d005bb82618dba P 086e23e953ee4ecdb9a68942c3037fd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:23:02.489822 31999 log.cc:1196] T 5cf7e5d1f2534894b2d005bb82618dba P 086e23e953ee4ecdb9a68942c3037fd5: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-2/wal/wals/5cf7e5d1f2534894b2d005bb82618dba
I20260808 18:23:02.490157 31999 ts_tablet_manager.cc:1939] T 5cf7e5d1f2534894b2d005bb82618dba P 086e23e953ee4ecdb9a68942c3037fd5: Deleting consensus metadata
I20260808 18:23:02.490990 31344 catalog_manager.cc:4841] TS 086e23e953ee4ecdb9a68942c3037fd5 (127.23.72.131:41097): tablet 5cf7e5d1f2534894b2d005bb82618dba (table test-table [id=0fd6aa66f585402d86419475432a4589]) successfully deleted
I20260808 18:23:02.492067 32001 ts_tablet_manager.cc:1918] T 5cf7e5d1f2534894b2d005bb82618dba P f9977432298448988bdb259803c1171b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:23:02.492173 32001 log.cc:1196] T 5cf7e5d1f2534894b2d005bb82618dba P f9977432298448988bdb259803c1171b: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213345431840-23842-0/minicluster-data/ts-1/wal/wals/5cf7e5d1f2534894b2d005bb82618dba
I20260808 18:23:02.492586 32001 ts_tablet_manager.cc:1939] T 5cf7e5d1f2534894b2d005bb82618dba P f9977432298448988bdb259803c1171b: Deleting consensus metadata
I20260808 18:23:02.493444 31341 catalog_manager.cc:4841] TS f9977432298448988bdb259803c1171b (127.23.72.130:33107): tablet 5cf7e5d1f2534894b2d005bb82618dba (table test-table [id=0fd6aa66f585402d86419475432a4589]) successfully deleted
I20260808 18:23:02.496641 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 31564
I20260808 18:23:02.497006 31968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:02.496649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:38285 (local address 127.0.0.1:57798)
0808 18:23:02.496823 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:02.496897 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.129:38285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260808 18:23:02.499223 31341 catalog_manager.cc:4584] TS 0c00472449774db482dac0a3c10272ed (127.23.72.129:38285): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5cf7e5d1f2534894b2d005bb82618dba: Network error: Client connection negotiation failed: client connection to 127.23.72.129:38285: connect: Connection refused (error 111)
I20260808 18:23:02.507742 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 31711
I20260808 18:23:02.517437 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 31000
I20260808 18:23:02.525329 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 31122
2026-08-08T18:23:02Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7773 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:23:02 dist-test-slave-5k54 krb5kdc[32004](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:23:02 dist-test-slave-5k54 krb5kdc[32004](info): set up 2 sockets
Aug 08 18:23:02 dist-test-slave-5k54 krb5kdc[32004](info): commencing operation
krb5kdc: starting...
W20260808 18:23:03.617481 23842 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Aug 08 18:23:03 dist-test-slave-5k54 krb5kdc[32004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:23:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:23:03Z Disabled control of system clock
WARNING: no policy specified for oryx/127.23.72.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.72.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.72.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.190.keytab.
Entry for principal oryx/127.23.72.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.190.keytab.
I20260808 18:23:03.748790 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:40515
--webserver_interface=127.23.72.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.190.keytab
--principal=oryx/127.23.72.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:40023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:40515
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:23:03.875130 32017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:23:03.875419 32017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:23:03.875533 32017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:23:03.879575 32017 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:23:03.879654 32017 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 18:23:03.879685 32017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:23:03.879729 32017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:23:03.879765 32017 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:23:03.879799 32017 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:23:03.885192 32017 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:40023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:40515
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.190.keytab
--principal=oryx/127.23.72.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:40515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.72.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:23:03.886263 32017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:23:03.887287 32017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:23:03.896807 32023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:23:03.896852 32026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:23:03.896998 32017 server_base.cc:1034] running on GCE node
W20260808 18:23:03.896807 32022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:23:03.897679 32017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:23:03.898727 32017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:23:03.899892 32017 hybrid_clock.cc:648] HybridClock initialized: now 1786213383899853 us; error 50 us; skew 500 ppm
Aug 08 18:23:03 dist-test-slave-5k54 krb5kdc[32004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213383, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.72.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:23:03.902621 32017 init.cc:377] Logged in from keytab as oryx/127.23.72.190@KRBTEST.COM (short username oryx)
I20260808 18:23:03.903596 32017 webserver.cc:460] Webserver started at http://127.23.72.190:39257/ using document root <none> and password file <none>
I20260808 18:23:03.904145 32017 fs_manager.cc:359] Metadata directory not provided
I20260808 18:23:03.904208 32017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:23:03.904433 32017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:23:03.906548 32017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "f0f38b8754bb4ca6a2ec3310e69c9c55"
format_stamp: "Formatted at 2026-08-08 18:23:03 on dist-test-slave-5k54"
server_key: "0d4e2a3a688dfa3f0c7dd653dbf8c270"
server_key_iv: "3171ca6c4665d33bee1fd6f9b367cedc"
server_key_version: "encryptionkey@0"
I20260808 18:23:03.907102 32017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "f0f38b8754bb4ca6a2ec3310e69c9c55"
format_stamp: "Formatted at 2026-08-08 18:23:03 on dist-test-slave-5k54"
server_key: "0d4e2a3a688dfa3f0c7dd653dbf8c270"
server_key_iv: "3171ca6c4665d33bee1fd6f9b367cedc"
server_key_version: "encryptionkey@0"
I20260808 18:23:03.910782 32017 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 18:23:03.913211 32033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:03.914430 32017 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260808 18:23:03.914569 32017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "f0f38b8754bb4ca6a2ec3310e69c9c55"
format_stamp: "Formatted at 2026-08-08 18:23:03 on dist-test-slave-5k54"
server_key: "0d4e2a3a688dfa3f0c7dd653dbf8c270"
server_key_iv: "3171ca6c4665d33bee1fd6f9b367cedc"
server_key_version: "encryptionkey@0"
I20260808 18:23:03.914670 32017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:23:03.935137 32017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:23:03.938544 32017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:23:03.938783 32017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:23:03.947868 32017 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.190:40515
I20260808 18:23:03.947898 32108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.190:40515 every 8 connection(s)
I20260808 18:23:03.948892 32017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:23:03.951924 32109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:23:03.956060 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 32017
I20260808 18:23:03.956205 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:23:03.956538 23842 external_mini_cluster.cc:1428] Setting key 2764001042a7d0152657fc79f1d2e85a
I20260808 18:23:03.957832 32109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0f38b8754bb4ca6a2ec3310e69c9c55: Bootstrap starting.
I20260808 18:23:03.960593 32109 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f0f38b8754bb4ca6a2ec3310e69c9c55: Neither blocks nor log segments found. Creating new log.
I20260808 18:23:03.961491 32109 log.cc:824] T 00000000000000000000000000000000 P f0f38b8754bb4ca6a2ec3310e69c9c55: Log is configured to *not* fsync() on all Append() calls
I20260808 18:23:03.963649 32109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0f38b8754bb4ca6a2ec3310e69c9c55: No bootstrap required, opened a new log
Aug 08 18:23:03 dist-test-slave-5k54 krb5kdc[32004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM
I20260808 18:23:03.968251 32109 raft_consensus.cc:348] T 00000000000000000000000000000000 P f0f38b8754bb4ca6a2ec3310e69c9c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0f38b8754bb4ca6a2ec3310e69c9c55" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40515 } }
I20260808 18:23:03.968528 32109 raft_consensus.cc:374] T 00000000000000000000000000000000 P f0f38b8754bb4ca6a2ec3310e69c9c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:23:03.968587 32109 raft_consensus.cc:729] T 00000000000000000000000000000000 P f0f38b8754bb4ca6a2ec3310e69c9c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0f38b8754bb4ca6a2ec3310e69c9c55, State: Initialized, Role: FOLLOWER
I20260808 18:23:03.969105 32109 consensus_queue.cc:260] T 00000000000000000000000000000000 P f0f38b8754bb4ca6a2ec3310e69c9c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0f38b8754bb4ca6a2ec3310e69c9c55" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40515 } }
I20260808 18:23:03.969305 32109 raft_consensus.cc:388] T 00000000000000000000000000000000 P f0f38b8754bb4ca6a2ec3310e69c9c55 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:23:03.969401 32109 raft_consensus.cc:482] T 00000000000000000000000000000000 P f0f38b8754bb4ca6a2ec3310e69c9c55 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:23:03.969506 32109 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f0f38b8754bb4ca6a2ec3310e69c9c55 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:23:03.970659 32109 raft_consensus.cc:504] T 00000000000000000000000000000000 P f0f38b8754bb4ca6a2ec3310e69c9c55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0f38b8754bb4ca6a2ec3310e69c9c55" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40515 } }
I20260808 18:23:03.971040 32109 leader_election.cc:302] T 00000000000000000000000000000000 P f0f38b8754bb4ca6a2ec3310e69c9c55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f0f38b8754bb4ca6a2ec3310e69c9c55; no voters:
I20260808 18:23:03.971475 32109 leader_election.cc:288] T 00000000000000000000000000000000 P f0f38b8754bb4ca6a2ec3310e69c9c55 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:23:03.971609 32115 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f0f38b8754bb4ca6a2ec3310e69c9c55 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:23:03.971917 32115 raft_consensus.cc:686] T 00000000000000000000000000000000 P f0f38b8754bb4ca6a2ec3310e69c9c55 [term 1 LEADER]: Becoming Leader. State: Replica: f0f38b8754bb4ca6a2ec3310e69c9c55, State: Running, Role: LEADER
I20260808 18:23:03.972286 32115 consensus_queue.cc:237] T 00000000000000000000000000000000 P f0f38b8754bb4ca6a2ec3310e69c9c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0f38b8754bb4ca6a2ec3310e69c9c55" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40515 } }
I20260808 18:23:03.972388 32113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:03.958075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35156 (local address 127.23.72.190:40515)
0808 18:23:03.958557 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:03.958578 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:03.958622 (+ 44us) server_negotiation.cc:407] Connection header received
0808 18:23:03.959523 (+ 901us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:03.959546 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:03.959965 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:03.960334 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:03.961156 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:03.962534 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:03.963405 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:03.964099 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:03.964260 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:03.966963 (+ 2703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:23:03.966970 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:23:03.966986 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:23:03.967017 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:23:03.969050 (+ 2033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:23:03.969538 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:23:03.969544 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:23:03.969551 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:23:03.969703 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:23:03.969991 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:23:03.969994 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:23:03.969996 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:23:03.970550 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:23:03.970740 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:03.971024 (+ 284us) server_negotiation.cc:299] Negotiation successful
0808 18:23:03.971376 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":325,"thread_start_us":139,"threads_started":1}
I20260808 18:23:03.972745 32109 sys_catalog.cc:564] T 00000000000000000000000000000000 P f0f38b8754bb4ca6a2ec3310e69c9c55 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:23:03.973973 32118 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0f38b8754bb4ca6a2ec3310e69c9c55 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f0f38b8754bb4ca6a2ec3310e69c9c55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0f38b8754bb4ca6a2ec3310e69c9c55" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40515 } } }
I20260808 18:23:03.974100 32118 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0f38b8754bb4ca6a2ec3310e69c9c55 [sys.catalog]: This master's current role is: LEADER
I20260808 18:23:03.974318 32117 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0f38b8754bb4ca6a2ec3310e69c9c55 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f0f38b8754bb4ca6a2ec3310e69c9c55. Latest consensus state: current_term: 1 leader_uuid: "f0f38b8754bb4ca6a2ec3310e69c9c55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0f38b8754bb4ca6a2ec3310e69c9c55" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40515 } } }
I20260808 18:23:03.974427 32117 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0f38b8754bb4ca6a2ec3310e69c9c55 [sys.catalog]: This master's current role is: LEADER
I20260808 18:23:03.975312 32125 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:23:03.978024 32125 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:23:03.983872 32125 catalog_manager.cc:1347] Generated new cluster ID: 8ebc61adbd4847e7bf916b449d70ccfc
I20260808 18:23:03.983963 32125 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:23:03.998116 32125 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:23:03.999405 32125 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:23:04.014137 32125 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f0f38b8754bb4ca6a2ec3310e69c9c55: Generated new TSK 0
I20260808 18:23:04.015035 32125 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.23.72.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.72.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.72.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.129.keytab.
Entry for principal oryx/127.23.72.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.129.keytab.
I20260808 18:23:04.055462 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.129:0
--local_ip_for_outbound_sockets=127.23.72.129
--webserver_interface=127.23.72.129
--webserver_port=0
--tserver_master_addrs=127.23.72.190:40515
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.129.keytab
--principal=oryx/127.23.72.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:40023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:23:04.174010 32139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:23:04.174286 32139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:23:04.174356 32139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:23:04.178493 32139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:23:04.178584 32139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:23:04.178653 32139 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 18:23:04.178737 32139 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.129
I20260808 18:23:04.184067 32139 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:40023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.129.keytab
--principal=oryx/127.23.72.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.72.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.72.190:40515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32139
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.129
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:23:04.185141 32139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:23:04.186173 32139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:23:04.195636 32144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:23:04.195642 32148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:23:04.195653 32145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:23:04.195883 32139 server_base.cc:1034] running on GCE node
I20260808 18:23:04.196306 32139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:23:04.196995 32139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:23:04.198230 32139 hybrid_clock.cc:648] HybridClock initialized: now 1786213384198217 us; error 33 us; skew 500 ppm
Aug 08 18:23:04 dist-test-slave-5k54 krb5kdc[32004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213384, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.72.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:23:04.201203 32139 init.cc:377] Logged in from keytab as oryx/127.23.72.129@KRBTEST.COM (short username oryx)
I20260808 18:23:04.202046 32139 webserver.cc:460] Webserver started at http://127.23.72.129:34511/ using document root <none> and password file <none>
I20260808 18:23:04.202569 32139 fs_manager.cc:359] Metadata directory not provided
I20260808 18:23:04.202629 32139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:23:04.202816 32139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:23:04.204985 32139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-0/data/instance:
uuid: "0aba19b7cb8a402a822a2fd608e3f697"
format_stamp: "Formatted at 2026-08-08 18:23:04 on dist-test-slave-5k54"
server_key: "c279544d3b93d1a986de52b8def0676d"
server_key_iv: "ffc5d750c722e33227304a21335d8edf"
server_key_version: "encryptionkey@0"
I20260808 18:23:04.205549 32139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance:
uuid: "0aba19b7cb8a402a822a2fd608e3f697"
format_stamp: "Formatted at 2026-08-08 18:23:04 on dist-test-slave-5k54"
server_key: "c279544d3b93d1a986de52b8def0676d"
server_key_iv: "ffc5d750c722e33227304a21335d8edf"
server_key_version: "encryptionkey@0"
I20260808 18:23:04.209309 32139 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260808 18:23:04.211750 32156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:04.212925 32139 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 18:23:04.213059 32139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-0/wal
uuid: "0aba19b7cb8a402a822a2fd608e3f697"
format_stamp: "Formatted at 2026-08-08 18:23:04 on dist-test-slave-5k54"
server_key: "c279544d3b93d1a986de52b8def0676d"
server_key_iv: "ffc5d750c722e33227304a21335d8edf"
server_key_version: "encryptionkey@0"
I20260808 18:23:04.213169 32139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:23:04.232417 32139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:23:04.235828 32139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:23:04.236042 32139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:23:04.237563 32139 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:23:04.237641 32139 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:04.237716 32139 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:23:04.237768 32139 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 18:23:04 dist-test-slave-5k54 krb5kdc[32004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213384, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.72.129@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM
I20260808 18:23:04.248808 32139 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.129:44001
I20260808 18:23:04.248895 32284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.129:44001 every 8 connection(s)
I20260808 18:23:04.249697 32139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
I20260808 18:23:04.252985 32113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:04.238534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:52253 (local address 127.23.72.190:40515)
0808 18:23:04.238710 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:04.238714 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:04.239644 (+ 930us) server_negotiation.cc:407] Connection header received
0808 18:23:04.240838 (+ 1194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:04.240841 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:04.240914 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:04.241046 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:04.242721 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:04.243607 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:04.245510 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:04.245843 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:04.245934 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:04.249247 (+ 3313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:23:04.249251 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:23:04.249253 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:23:04.249285 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:23:04.250950 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:23:04.251558 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:23:04.251561 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:23:04.251562 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:23:04.251640 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:23:04.252004 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:23:04.252006 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:23:04.252008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:23:04.252179 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:23:04.252252 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:04.252696 (+ 444us) server_negotiation.cc:299] Negotiation successful
0808 18:23:04.252812 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 18:23:04.253775 32166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:04.238928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:40515 (local address 127.23.72.129:52253)
0808 18:23:04.239484 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:04.239535 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:04.240538 (+ 1003us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:04.241271 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:04.241283 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:04.241778 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:04.242530 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:04.242555 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:04.243773 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:04.243777 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:04.245368 (+ 1591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:04.245382 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:04.246016 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:04.246020 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:04.246262 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:04.247067 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:04.247092 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:04.249055 (+ 1963us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:23:04.251071 (+ 2016us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:23:04.251077 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:23:04.251091 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:23:04.251466 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:23:04.251726 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:23:04.251728 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:23:04.251730 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:23:04.251910 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:23:04.252267 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:23:04.252273 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:23:04.252484 (+ 211us) client_negotiation.cc:769] Sending connection context
0808 18:23:04.252628 (+ 144us) client_negotiation.cc:240] Negotiation successful
0808 18:23:04.252918 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":150,"threads_started":1}
I20260808 18:23:04.253808 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 32139
I20260808 18:23:04.254019 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance
I20260808 18:23:04.254360 23842 external_mini_cluster.cc:1428] Setting key e8537e6711b9fb83acf47892f4da4d47
I20260808 18:23:04.255328 32285 heartbeater.cc:347] Connected to a master server at 127.23.72.190:40515
I20260808 18:23:04.255506 32285 heartbeater.cc:464] Registering TS with master...
I20260808 18:23:04.256037 32285 heartbeater.cc:511] Master 127.23.72.190:40515 requested a full tablet report, sending...
I20260808 18:23:04.257706 32051 ts_manager.cc:194] Registered new tserver with Master: 0aba19b7cb8a402a822a2fd608e3f697 (127.23.72.129:44001)
I20260808 18:23:04.259039 32051 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.72.129@KRBTEST.COM'} at 127.23.72.129:52253
WARNING: no policy specified for oryx/127.23.72.130@KRBTEST.COM; defaulting to no policy
I20260808 18:23:04.265549 32113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:04.257620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:43885 (local address 127.23.72.190:40515)
0808 18:23:04.257732 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:04.257736 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:04.257978 (+ 242us) server_negotiation.cc:407] Connection header received
0808 18:23:04.258090 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:04.258093 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:04.258160 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:04.258240 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:04.258889 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:04.259605 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:04.260643 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:04.260888 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:04.261004 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:04.262144 (+ 1140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:23:04.262148 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:23:04.262151 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:23:04.262174 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:23:04.263855 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:23:04.264316 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:23:04.264321 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:23:04.264322 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:23:04.264404 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:23:04.264699 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:23:04.264704 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:23:04.264706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:23:04.264891 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:23:04.265009 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:04.265267 (+ 258us) server_negotiation.cc:299] Negotiation successful
0808 18:23:04.265377 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 18:23:04.265563 32292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:04.257563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:40515 (local address 127.23.72.129:43885)
0808 18:23:04.257895 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:04.257907 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:04.258015 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:04.258381 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:04.258386 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:04.258618 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:04.258753 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:04.258763 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:04.259938 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:04.259941 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:04.260520 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:04.260527 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:04.261010 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:04.261014 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:04.261103 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:04.261662 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:04.261674 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:04.262006 (+ 332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:23:04.263983 (+ 1977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:23:04.263986 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:23:04.263988 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:23:04.264211 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:23:04.264511 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:23:04.264513 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:23:04.264515 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:23:04.264607 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:23:04.264999 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:23:04.265003 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:23:04.265105 (+ 102us) client_negotiation.cc:769] Sending connection context
0808 18:23:04.265213 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 18:23:04.265367 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"mutex_wait_us":3,"thread_start_us":97,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.72.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.72.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.130.keytab.
Entry for principal oryx/127.23.72.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.130.keytab.
I20260808 18:23:04.286607 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.130:0
--local_ip_for_outbound_sockets=127.23.72.130
--webserver_interface=127.23.72.130
--webserver_port=0
--tserver_master_addrs=127.23.72.190:40515
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.130.keytab
--principal=oryx/127.23.72.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:40023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:23:04.401175 32295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:23:04.401428 32295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:23:04.401472 32295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:23:04.405839 32295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:23:04.405900 32295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:23:04.405938 32295 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 18:23:04.406016 32295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.130
I20260808 18:23:04.411164 32295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:40023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.130.keytab
--principal=oryx/127.23.72.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.72.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.72.190:40515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.130
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:23:04.412220 32295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:23:04.413275 32295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:23:04.423379 32301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:23:04.423437 32300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:23:04.423588 32303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:23:04.423588 32295 server_base.cc:1034] running on GCE node
I20260808 18:23:04.424136 32295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:23:04.424804 32295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:23:04.425993 32295 hybrid_clock.cc:648] HybridClock initialized: now 1786213384425981 us; error 31 us; skew 500 ppm
Aug 08 18:23:04 dist-test-slave-5k54 krb5kdc[32004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213384, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.72.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:23:04.428946 32295 init.cc:377] Logged in from keytab as oryx/127.23.72.130@KRBTEST.COM (short username oryx)
I20260808 18:23:04.429890 32295 webserver.cc:460] Webserver started at http://127.23.72.130:46663/ using document root <none> and password file <none>
I20260808 18:23:04.430428 32295 fs_manager.cc:359] Metadata directory not provided
I20260808 18:23:04.430488 32295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:23:04.430655 32295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:23:04.432703 32295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-1/data/instance:
uuid: "6bbded853ce94d80b70ff5059443baca"
format_stamp: "Formatted at 2026-08-08 18:23:04 on dist-test-slave-5k54"
server_key: "fa3c85987d7c1dfc879dd0f5f7b09c55"
server_key_iv: "eb0600238116f8199f042ae50a95effb"
server_key_version: "encryptionkey@0"
I20260808 18:23:04.433246 32295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance:
uuid: "6bbded853ce94d80b70ff5059443baca"
format_stamp: "Formatted at 2026-08-08 18:23:04 on dist-test-slave-5k54"
server_key: "fa3c85987d7c1dfc879dd0f5f7b09c55"
server_key_iv: "eb0600238116f8199f042ae50a95effb"
server_key_version: "encryptionkey@0"
I20260808 18:23:04.436914 32295 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:23:04.439354 32310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:04.440725 32295 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:23:04.440831 32295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-1/wal
uuid: "6bbded853ce94d80b70ff5059443baca"
format_stamp: "Formatted at 2026-08-08 18:23:04 on dist-test-slave-5k54"
server_key: "fa3c85987d7c1dfc879dd0f5f7b09c55"
server_key_iv: "eb0600238116f8199f042ae50a95effb"
server_key_version: "encryptionkey@0"
I20260808 18:23:04.440989 32295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:23:04.471181 32295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:23:04.475386 32295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:23:04.475706 32295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:23:04.477270 32295 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:23:04.477322 32295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:04.477398 32295 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:23:04.477434 32295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 18:23:04 dist-test-slave-5k54 krb5kdc[32004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213384, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.72.130@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM
I20260808 18:23:04.489357 32295 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.130:43777
I20260808 18:23:04.489420 32435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.130:43777 every 8 connection(s)
I20260808 18:23:04.490590 32295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
I20260808 18:23:04.492798 32113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:04.478113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:59007 (local address 127.23.72.190:40515)
0808 18:23:04.478269 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:04.478273 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:04.479187 (+ 914us) server_negotiation.cc:407] Connection header received
0808 18:23:04.480392 (+ 1205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:04.480398 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:04.480459 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:04.480522 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:04.482316 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:04.483308 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:04.484676 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:04.484982 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:04.485110 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:04.488336 (+ 3226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:23:04.488339 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:23:04.488341 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:23:04.488367 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:23:04.490298 (+ 1931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:23:04.490972 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:23:04.490978 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:23:04.490981 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:23:04.491105 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:23:04.491606 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:23:04.491610 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:23:04.491612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:23:04.491818 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:23:04.491953 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:04.492475 (+ 522us) server_negotiation.cc:299] Negotiation successful
0808 18:23:04.492611 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 18:23:04.493737 32319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:04.478549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:40515 (local address 127.23.72.130:59007)
0808 18:23:04.479017 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:04.479066 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:04.480119 (+ 1053us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:04.480968 (+ 849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:04.480977 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:04.481504 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:04.482151 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:04.482175 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:04.483496 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:04.483500 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:04.484539 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:04.484549 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:04.485160 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:04.485164 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:04.485343 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:04.485991 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:04.486010 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:04.488130 (+ 2120us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:23:04.490444 (+ 2314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:23:04.490449 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:23:04.490461 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:23:04.490829 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:23:04.491234 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:23:04.491237 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:23:04.491239 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:23:04.491477 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:23:04.491947 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:23:04.491953 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:23:04.492206 (+ 253us) client_negotiation.cc:769] Sending connection context
0808 18:23:04.492407 (+ 201us) client_negotiation.cc:240] Negotiation successful
0808 18:23:04.492737 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":133,"threads_started":1}
I20260808 18:23:04.494184 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 32295
I20260808 18:23:04.494326 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance
I20260808 18:23:04.494702 23842 external_mini_cluster.cc:1428] Setting key d016afb2575637d6adb7fadfdd9ab67f
I20260808 18:23:04.495182 32436 heartbeater.cc:347] Connected to a master server at 127.23.72.190:40515
I20260808 18:23:04.495337 32436 heartbeater.cc:464] Registering TS with master...
I20260808 18:23:04.495936 32436 heartbeater.cc:511] Master 127.23.72.190:40515 requested a full tablet report, sending...
I20260808 18:23:04.497044 32051 ts_manager.cc:194] Registered new tserver with Master: 6bbded853ce94d80b70ff5059443baca (127.23.72.130:43777)
I20260808 18:23:04.497771 32051 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.72.130@KRBTEST.COM'} at 127.23.72.130:59007
I20260808 18:23:04.505695 32443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:04.497433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:40515 (local address 127.23.72.130:35129)
0808 18:23:04.498157 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:04.498170 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:04.498288 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:04.498707 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:04.498710 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:04.498908 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:04.499006 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:04.499011 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:04.499922 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:04.499926 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:04.500535 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:04.500544 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:04.501133 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:04.501137 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:04.501262 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:04.501886 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:04.501898 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:04.502215 (+ 317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:23:04.504038 (+ 1823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:23:04.504045 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:23:04.504048 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:23:04.504262 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:23:04.504615 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:23:04.504618 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:23:04.504619 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:23:04.504711 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:23:04.505265 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:23:04.505268 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:23:04.505349 (+ 81us) client_negotiation.cc:769] Sending connection context
0808 18:23:04.505442 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 18:23:04.505553 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":634,"thread_start_us":114,"threads_started":1}
WARNING: no policy specified for oryx/127.23.72.131@KRBTEST.COM; defaulting to no policy
I20260808 18:23:04.505847 32113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:04.497483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:35129 (local address 127.23.72.190:40515)
0808 18:23:04.497584 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:04.497588 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:04.498248 (+ 660us) server_negotiation.cc:407] Connection header received
0808 18:23:04.498398 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:04.498405 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:04.498468 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:04.498565 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:04.499130 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:04.499783 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:04.500662 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:04.500988 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:04.501118 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:04.502382 (+ 1264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:23:04.502385 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:23:04.502388 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:23:04.502421 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:23:04.503925 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:23:04.504408 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:23:04.504412 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:23:04.504413 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:23:04.504504 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:23:04.504829 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:23:04.504836 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:23:04.504839 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:23:04.505151 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:23:04.505274 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:04.505534 (+ 260us) server_negotiation.cc:299] Negotiation successful
0808 18:23:04.505645 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.72.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.72.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.131.keytab.
Entry for principal oryx/127.23.72.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.131.keytab.
I20260808 18:23:04.524780 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.131:0
--local_ip_for_outbound_sockets=127.23.72.131
--webserver_interface=127.23.72.131
--webserver_port=0
--tserver_master_addrs=127.23.72.190:40515
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.131.keytab
--principal=oryx/127.23.72.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:40023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:23:04.635274 32447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:23:04.635589 32447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:23:04.635639 32447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:23:04.639580 32447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:23:04.639631 32447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:23:04.639667 32447 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 18:23:04.639699 32447 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.131
I20260808 18:23:04.644811 32447 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:40023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.131.keytab
--principal=oryx/127.23.72.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.72.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.72.190:40515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32447
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.131
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:23:04.645817 32447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:23:04.646770 32447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:23:04.656201 32453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:23:04.656272 32457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:23:04.656245 32455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:23:04.656512 32447 server_base.cc:1034] running on GCE node
I20260808 18:23:04.656920 32447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:23:04.657610 32447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:23:04.658802 32447 hybrid_clock.cc:648] HybridClock initialized: now 1786213384658752 us; error 71 us; skew 500 ppm
Aug 08 18:23:04 dist-test-slave-5k54 krb5kdc[32004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213384, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.72.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:23:04.661751 32447 init.cc:377] Logged in from keytab as oryx/127.23.72.131@KRBTEST.COM (short username oryx)
I20260808 18:23:04.662642 32447 webserver.cc:460] Webserver started at http://127.23.72.131:35245/ using document root <none> and password file <none>
I20260808 18:23:04.663193 32447 fs_manager.cc:359] Metadata directory not provided
I20260808 18:23:04.663286 32447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:23:04.663587 32447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:23:04.665592 32447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-2/data/instance:
uuid: "b340acac117a4ec8a029ab8195ea42ba"
format_stamp: "Formatted at 2026-08-08 18:23:04 on dist-test-slave-5k54"
server_key: "e0bf08be6033b495a545cbb23cc23973"
server_key_iv: "0f361f3dc517e602bf576a99e3aaf58d"
server_key_version: "encryptionkey@0"
I20260808 18:23:04.666319 32447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance:
uuid: "b340acac117a4ec8a029ab8195ea42ba"
format_stamp: "Formatted at 2026-08-08 18:23:04 on dist-test-slave-5k54"
server_key: "e0bf08be6033b495a545cbb23cc23973"
server_key_iv: "0f361f3dc517e602bf576a99e3aaf58d"
server_key_version: "encryptionkey@0"
I20260808 18:23:04.670168 32447 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 18:23:04.672844 32465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:04.674203 32447 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:23:04.674328 32447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-2/wal
uuid: "b340acac117a4ec8a029ab8195ea42ba"
format_stamp: "Formatted at 2026-08-08 18:23:04 on dist-test-slave-5k54"
server_key: "e0bf08be6033b495a545cbb23cc23973"
server_key_iv: "0f361f3dc517e602bf576a99e3aaf58d"
server_key_version: "encryptionkey@0"
I20260808 18:23:04.674448 32447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:23:04.697032 32447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:23:04.700412 32447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:23:04.700618 32447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:23:04.702121 32447 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:23:04.702193 32447 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:04.702265 32447 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:23:04.702315 32447 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 18:23:04 dist-test-slave-5k54 krb5kdc[32004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213384, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.72.131@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM
I20260808 18:23:04.713625 32447 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.131:41059
I20260808 18:23:04.713701 32599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.131:41059 every 8 connection(s)
I20260808 18:23:04.714535 32447 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
I20260808 18:23:04.716666 32113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:04.703111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:54979 (local address 127.23.72.190:40515)
0808 18:23:04.703245 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:04.703249 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:04.704336 (+ 1087us) server_negotiation.cc:407] Connection header received
0808 18:23:04.705555 (+ 1219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:04.705562 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:04.705640 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:04.705749 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:04.707026 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:04.707744 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:04.709162 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:04.709481 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:04.709617 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:04.712976 (+ 3359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:23:04.712979 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:23:04.712982 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:23:04.713019 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:23:04.714578 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:23:04.715105 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:23:04.715110 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:23:04.715111 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:23:04.715206 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:23:04.715631 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:23:04.715633 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:23:04.715635 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:23:04.715828 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:23:04.715914 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:04.716415 (+ 501us) server_negotiation.cc:299] Negotiation successful
0808 18:23:04.716516 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 18:23:04.717437 32476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:04.703700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:40515 (local address 127.23.72.131:54979)
0808 18:23:04.704179 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:04.704227 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:04.705246 (+ 1019us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:04.705963 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:04.705970 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:04.706280 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:04.706857 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:04.706877 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:04.707878 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:04.707881 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:04.709028 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:04.709038 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:04.709665 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:04.709670 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:04.709876 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:04.710478 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:04.710496 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:04.712810 (+ 2314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:23:04.714708 (+ 1898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:23:04.714714 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:23:04.714727 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:23:04.715008 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:23:04.715326 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:23:04.715329 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:23:04.715331 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:23:04.715541 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:23:04.715949 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:23:04.715955 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:23:04.716175 (+ 220us) client_negotiation.cc:769] Sending connection context
0808 18:23:04.716329 (+ 154us) client_negotiation.cc:240] Negotiation successful
0808 18:23:04.716557 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":139,"threads_started":1}
I20260808 18:23:04.718636 32600 heartbeater.cc:347] Connected to a master server at 127.23.72.190:40515
I20260808 18:23:04.718787 32600 heartbeater.cc:464] Registering TS with master...
I20260808 18:23:04.719305 32600 heartbeater.cc:511] Master 127.23.72.190:40515 requested a full tablet report, sending...
I20260808 18:23:04.720391 32052 ts_manager.cc:194] Registered new tserver with Master: b340acac117a4ec8a029ab8195ea42ba (127.23.72.131:41059)
I20260808 18:23:04.721230 32052 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.72.131@KRBTEST.COM'} at 127.23.72.131:54979
I20260808 18:23:04.722079 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 32447
I20260808 18:23:04.722223 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance
I20260808 18:23:04.722640 23842 external_mini_cluster.cc:1428] Setting key ca9522944a199ebf8f6fe19816e81359
I20260808 18:23:04.724903 23842 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:23:04.728495 32113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:04.720903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:49931 (local address 127.23.72.190:40515)
0808 18:23:04.721017 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:04.721020 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:04.721193 (+ 173us) server_negotiation.cc:407] Connection header received
0808 18:23:04.721301 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:04.721303 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:04.721388 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:04.721453 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:04.721972 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:04.722624 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:04.723924 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:04.724204 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:04.724315 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:04.725402 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:23:04.725406 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:23:04.725408 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:23:04.725432 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:23:04.726859 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:23:04.727288 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:23:04.727291 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:23:04.727293 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:23:04.727383 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:23:04.727706 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:23:04.727710 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:23:04.727712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:23:04.727884 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:23:04.727988 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:04.728242 (+ 254us) server_negotiation.cc:299] Negotiation successful
0808 18:23:04.728360 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 18:23:04.728506 32608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:04.720819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:40515 (local address 127.23.72.131:49931)
0808 18:23:04.721113 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:04.721127 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:04.721221 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:04.721478 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:04.721484 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:04.721721 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:04.721860 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:04.721866 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:04.722810 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:04.722813 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:04.723787 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:04.723804 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:04.724316 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:04.724320 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:04.724394 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:04.724893 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:04.724913 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:04.725273 (+ 360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:23:04.726979 (+ 1706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:23:04.726982 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:23:04.726984 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:23:04.727180 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:23:04.727520 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:23:04.727523 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:23:04.727525 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:23:04.727614 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:23:04.727979 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:23:04.727982 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:23:04.728079 (+ 97us) client_negotiation.cc:769] Sending connection context
0808 18:23:04.728186 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 18:23:04.728325 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":93,"threads_started":1}
Aug 08 18:23:04 dist-test-slave-5k54 krb5kdc[32004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213384, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 18:23:04 dist-test-slave-5k54 krb5kdc[32004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:04 dist-test-slave-5k54 krb5kdc[32004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 18:23:04.750375 32113 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:04.744290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36934 (local address 127.23.72.190:40515)
0808 18:23:04.744432 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:04.744435 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:04.744448 (+ 13us) server_negotiation.cc:407] Connection header received
0808 18:23:04.744489 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:04.744491 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:04.744528 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:04.744642 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:04.745436 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:04.746013 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:04.746825 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:04.747030 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:04.747103 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:04.750255 (+ 3152us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36934: BlockingRecv error: recv got EOF from 127.0.0.1:36934 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 08 18:23:04 dist-test-slave-5k54 krb5kdc[32004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213384, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM
I20260808 18:23:04.759769 32113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:04.752010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36944 (local address 127.23.72.190:40515)
0808 18:23:04.752133 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:04.752136 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:04.752241 (+ 105us) server_negotiation.cc:407] Connection header received
0808 18:23:04.752433 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:04.752436 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:04.752481 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:04.752551 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:04.753106 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:04.753588 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:04.754408 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:04.754605 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:04.754682 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:04.757114 (+ 2432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:23:04.757117 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:23:04.757119 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:23:04.757140 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:23:04.758359 (+ 1219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:23:04.758759 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:23:04.758762 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:23:04.758763 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:23:04.758838 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:23:04.759087 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:23:04.759089 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:23:04.759090 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:23:04.759243 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:23:04.759355 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:04.759570 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 18:23:04.759656 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 18:23:04.762512 32052 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36944:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:23:04.765583 32052 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:23:04.780547 32626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:04.774309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.131:41059 (local address 127.0.0.1:53204)
0808 18:23:04.774828 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:04.774879 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:04.775017 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:04.776128 (+ 1111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:04.776136 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:04.776165 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:23:04.776296 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:04.776316 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:04.777474 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:04.777478 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:04.779172 (+ 1694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:04.779181 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:04.780020 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:04.780024 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:04.780130 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:04.780165 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 18:23:04.780280 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 18:23:04.780409 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":144,"thread_start_us":64,"threads_started":1}
I20260808 18:23:04.780558 32627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:04.774540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53204 (local address 127.23.72.131:41059)
0808 18:23:04.775580 (+ 1040us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:04.775593 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:04.775619 (+ 26us) server_negotiation.cc:407] Connection header received
0808 18:23:04.775694 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:04.775698 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:04.775959 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:04.776099 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:23:04.776439 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:04.777363 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:04.779267 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:04.779898 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:04.780020 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:04.780192 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:04.780306 (+ 114us) server_negotiation.cc:299] Negotiation successful
0808 18:23:04.780401 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":905,"thread_start_us":119,"threads_started":1}
I20260808 18:23:04.780747 32629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:04.774699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:44001 (local address 127.0.0.1:54354)
0808 18:23:04.775280 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:04.775295 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:04.775392 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:04.776225 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:04.776231 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:04.776254 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:23:04.776365 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:04.776374 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:04.777698 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:04.777701 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:04.779074 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:04.779087 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:04.780291 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:04.780295 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:04.780408 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:04.780431 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 18:23:04.780497 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 18:23:04.780574 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"mutex_wait_us":94,"thread_start_us":80,"threads_started":1}
I20260808 18:23:04.781219 32628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:04.775053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.130:43777 (local address 127.0.0.1:57436)
0808 18:23:04.776344 (+ 1291us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:04.776357 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:04.776447 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:04.776947 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:04.776950 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:04.776969 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:23:04.777048 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:04.777053 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:04.778283 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:04.778286 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:04.779388 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:04.779399 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:04.780756 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:04.780763 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:04.780954 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:04.780979 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 18:23:04.781053 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 18:23:04.781111 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1230,"mutex_wait_us":27,"thread_start_us":99,"threads_started":1}
I20260808 18:23:04.781214 32630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:04.775133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54354 (local address 127.23.72.129:44001)
0808 18:23:04.775734 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:04.775744 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:04.775769 (+ 25us) server_negotiation.cc:407] Connection header received
0808 18:23:04.775861 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:04.775868 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:04.776072 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:04.776231 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:23:04.776500 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:04.777584 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:04.779189 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:04.779994 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:04.780095 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:04.780224 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:04.780981 (+ 757us) server_negotiation.cc:299] Negotiation successful
0808 18:23:04.781085 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":83,"threads_started":1}
I20260808 18:23:04.781309 32632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:04.775250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57436 (local address 127.23.72.130:43777)
0808 18:23:04.775719 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:04.775730 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:04.776543 (+ 813us) server_negotiation.cc:407] Connection header received
0808 18:23:04.776622 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:04.776626 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:04.776789 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:04.776946 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:23:04.777172 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:04.778158 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:04.779543 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:04.780322 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:04.780415 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:04.780560 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:04.781086 (+ 526us) server_negotiation.cc:299] Negotiation successful
0808 18:23:04.781177 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":114,"threads_started":1}
I20260808 18:23:04.783663 32213 tablet_service.cc:1467] Processing CreateTablet for tablet f9c5255c35ee4e10b88c256de01352bf (DEFAULT_TABLE table=test-table [id=bf157dce75c1470986865d4c18cfadfe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:23:04.783730 32524 tablet_service.cc:1467] Processing CreateTablet for tablet f9c5255c35ee4e10b88c256de01352bf (DEFAULT_TABLE table=test-table [id=bf157dce75c1470986865d4c18cfadfe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:23:04.783912 32362 tablet_service.cc:1467] Processing CreateTablet for tablet f9c5255c35ee4e10b88c256de01352bf (DEFAULT_TABLE table=test-table [id=bf157dce75c1470986865d4c18cfadfe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:23:04.784672 32213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9c5255c35ee4e10b88c256de01352bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:23:04.784739 32362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9c5255c35ee4e10b88c256de01352bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:23:04.784739 32524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9c5255c35ee4e10b88c256de01352bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:23:04.790527 32633 tablet_bootstrap.cc:492] T f9c5255c35ee4e10b88c256de01352bf P 0aba19b7cb8a402a822a2fd608e3f697: Bootstrap starting.
I20260808 18:23:04.792958 32634 tablet_bootstrap.cc:492] T f9c5255c35ee4e10b88c256de01352bf P b340acac117a4ec8a029ab8195ea42ba: Bootstrap starting.
I20260808 18:23:04.793365 32633 tablet_bootstrap.cc:654] T f9c5255c35ee4e10b88c256de01352bf P 0aba19b7cb8a402a822a2fd608e3f697: Neither blocks nor log segments found. Creating new log.
I20260808 18:23:04.794139 32633 log.cc:824] T f9c5255c35ee4e10b88c256de01352bf P 0aba19b7cb8a402a822a2fd608e3f697: Log is configured to *not* fsync() on all Append() calls
I20260808 18:23:04.794741 32634 tablet_bootstrap.cc:654] T f9c5255c35ee4e10b88c256de01352bf P b340acac117a4ec8a029ab8195ea42ba: Neither blocks nor log segments found. Creating new log.
I20260808 18:23:04.794781 32635 tablet_bootstrap.cc:492] T f9c5255c35ee4e10b88c256de01352bf P 6bbded853ce94d80b70ff5059443baca: Bootstrap starting.
I20260808 18:23:04.795480 32634 log.cc:824] T f9c5255c35ee4e10b88c256de01352bf P b340acac117a4ec8a029ab8195ea42ba: Log is configured to *not* fsync() on all Append() calls
I20260808 18:23:04.796986 32635 tablet_bootstrap.cc:654] T f9c5255c35ee4e10b88c256de01352bf P 6bbded853ce94d80b70ff5059443baca: Neither blocks nor log segments found. Creating new log.
I20260808 18:23:04.797093 32633 tablet_bootstrap.cc:492] T f9c5255c35ee4e10b88c256de01352bf P 0aba19b7cb8a402a822a2fd608e3f697: No bootstrap required, opened a new log
I20260808 18:23:04.797142 32634 tablet_bootstrap.cc:492] T f9c5255c35ee4e10b88c256de01352bf P b340acac117a4ec8a029ab8195ea42ba: No bootstrap required, opened a new log
I20260808 18:23:04.797318 32633 ts_tablet_manager.cc:1397] T f9c5255c35ee4e10b88c256de01352bf P 0aba19b7cb8a402a822a2fd608e3f697: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260808 18:23:04.797330 32634 ts_tablet_manager.cc:1397] T f9c5255c35ee4e10b88c256de01352bf P b340acac117a4ec8a029ab8195ea42ba: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 18:23:04.797844 32635 log.cc:824] T f9c5255c35ee4e10b88c256de01352bf P 6bbded853ce94d80b70ff5059443baca: Log is configured to *not* fsync() on all Append() calls
I20260808 18:23:04.800048 32635 tablet_bootstrap.cc:492] T f9c5255c35ee4e10b88c256de01352bf P 6bbded853ce94d80b70ff5059443baca: No bootstrap required, opened a new log
I20260808 18:23:04.800257 32635 ts_tablet_manager.cc:1397] T f9c5255c35ee4e10b88c256de01352bf P 6bbded853ce94d80b70ff5059443baca: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 18:23:04.800634 32634 raft_consensus.cc:348] T f9c5255c35ee4e10b88c256de01352bf P b340acac117a4ec8a029ab8195ea42ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } }
I20260808 18:23:04.800858 32634 raft_consensus.cc:374] T f9c5255c35ee4e10b88c256de01352bf P b340acac117a4ec8a029ab8195ea42ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:23:04.800908 32634 raft_consensus.cc:729] T f9c5255c35ee4e10b88c256de01352bf P b340acac117a4ec8a029ab8195ea42ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b340acac117a4ec8a029ab8195ea42ba, State: Initialized, Role: FOLLOWER
I20260808 18:23:04.801330 32634 consensus_queue.cc:260] T f9c5255c35ee4e10b88c256de01352bf P b340acac117a4ec8a029ab8195ea42ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } }
I20260808 18:23:04.801987 32600 heartbeater.cc:503] Master 127.23.72.190:40515 was elected leader, sending a full tablet report...
I20260808 18:23:04.802244 32634 ts_tablet_manager.cc:1428] T f9c5255c35ee4e10b88c256de01352bf P b340acac117a4ec8a029ab8195ea42ba: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
I20260808 18:23:04.802384 32633 raft_consensus.cc:348] T f9c5255c35ee4e10b88c256de01352bf P 0aba19b7cb8a402a822a2fd608e3f697 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } }
I20260808 18:23:04.802640 32633 raft_consensus.cc:374] T f9c5255c35ee4e10b88c256de01352bf P 0aba19b7cb8a402a822a2fd608e3f697 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:23:04.802706 32633 raft_consensus.cc:729] T f9c5255c35ee4e10b88c256de01352bf P 0aba19b7cb8a402a822a2fd608e3f697 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0aba19b7cb8a402a822a2fd608e3f697, State: Initialized, Role: FOLLOWER
I20260808 18:23:04.803215 32633 consensus_queue.cc:260] T f9c5255c35ee4e10b88c256de01352bf P 0aba19b7cb8a402a822a2fd608e3f697 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } }
I20260808 18:23:04.803885 32285 heartbeater.cc:503] Master 127.23.72.190:40515 was elected leader, sending a full tablet report...
I20260808 18:23:04.804248 32633 ts_tablet_manager.cc:1428] T f9c5255c35ee4e10b88c256de01352bf P 0aba19b7cb8a402a822a2fd608e3f697: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260808 18:23:04.806262 32635 raft_consensus.cc:348] T f9c5255c35ee4e10b88c256de01352bf P 6bbded853ce94d80b70ff5059443baca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } }
I20260808 18:23:04.806491 32635 raft_consensus.cc:374] T f9c5255c35ee4e10b88c256de01352bf P 6bbded853ce94d80b70ff5059443baca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:23:04.806557 32635 raft_consensus.cc:729] T f9c5255c35ee4e10b88c256de01352bf P 6bbded853ce94d80b70ff5059443baca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bbded853ce94d80b70ff5059443baca, State: Initialized, Role: FOLLOWER
I20260808 18:23:04.807032 32635 consensus_queue.cc:260] T f9c5255c35ee4e10b88c256de01352bf P 6bbded853ce94d80b70ff5059443baca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } }
I20260808 18:23:04.807786 32436 heartbeater.cc:503] Master 127.23.72.190:40515 was elected leader, sending a full tablet report...
I20260808 18:23:04.808167 32635 ts_tablet_manager.cc:1428] T f9c5255c35ee4e10b88c256de01352bf P 6bbded853ce94d80b70ff5059443baca: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
W20260808 18:23:04.966063 32601 tablet.cc:2369] T f9c5255c35ee4e10b88c256de01352bf P b340acac117a4ec8a029ab8195ea42ba: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 18:23:04.991945 32437 tablet.cc:2369] T f9c5255c35ee4e10b88c256de01352bf P 6bbded853ce94d80b70ff5059443baca: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 18:23:05.001299 32286 tablet.cc:2369] T f9c5255c35ee4e10b88c256de01352bf P 0aba19b7cb8a402a822a2fd608e3f697: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:23:05.086330 32641 raft_consensus.cc:482] T f9c5255c35ee4e10b88c256de01352bf P 0aba19b7cb8a402a822a2fd608e3f697 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:23:05.086602 32641 raft_consensus.cc:504] T f9c5255c35ee4e10b88c256de01352bf P 0aba19b7cb8a402a822a2fd608e3f697 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } }
I20260808 18:23:05.087863 32641 leader_election.cc:288] T f9c5255c35ee4e10b88c256de01352bf P 0aba19b7cb8a402a822a2fd608e3f697 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b340acac117a4ec8a029ab8195ea42ba (127.23.72.131:41059), 6bbded853ce94d80b70ff5059443baca (127.23.72.130:43777)
I20260808 18:23:05.092329 32627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:05.088274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:44853 (local address 127.23.72.131:41059)
0808 18:23:05.088457 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:05.088463 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:05.088587 (+ 124us) server_negotiation.cc:407] Connection header received
0808 18:23:05.088942 (+ 355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:05.088945 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:05.088997 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:05.089088 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:23:05.089438 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:05.090081 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:05.091289 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:05.091760 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:05.091890 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:05.091944 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:05.092120 (+ 176us) server_negotiation.cc:299] Negotiation successful
0808 18:23:05.092197 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260808 18:23:05.092353 32632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:05.088804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:59831 (local address 127.23.72.130:43777)
0808 18:23:05.088939 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:05.088942 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:05.089124 (+ 182us) server_negotiation.cc:407] Connection header received
0808 18:23:05.089219 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:05.089222 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:05.089271 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:05.089372 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:23:05.089681 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:05.090186 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:05.091345 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:05.091780 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:05.091914 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:05.091966 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:05.092173 (+ 207us) server_negotiation.cc:299] Negotiation successful
0808 18:23:05.092242 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 18:23:05.092344 32644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:05.088166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.131:41059 (local address 127.23.72.129:44853)
0808 18:23:05.088480 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:05.088493 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:05.088634 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:05.089131 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:05.089134 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:05.089159 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:23:05.089301 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:05.089312 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:05.090200 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:05.090203 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:05.091163 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:05.091172 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:05.091886 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:05.091889 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:05.091987 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:05.092011 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 18:23:05.092080 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 18:23:05.092162 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"mutex_wait_us":47,"thread_start_us":108,"threads_started":1}
I20260808 18:23:05.092360 32645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:05.088662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.130:43777 (local address 127.23.72.129:59831)
0808 18:23:05.089034 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:05.089049 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:05.089138 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:05.089432 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:05.089435 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:05.089454 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:23:05.089567 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:05.089573 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:05.090294 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:05.090297 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:05.091189 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:05.091198 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:05.091917 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:05.091921 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:05.092030 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:05.092046 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 18:23:05.092141 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 18:23:05.092202 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"mutex_wait_us":43,"thread_start_us":82,"threads_started":1}
I20260808 18:23:05.092978 32546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c5255c35ee4e10b88c256de01352bf" candidate_uuid: "0aba19b7cb8a402a822a2fd608e3f697" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b340acac117a4ec8a029ab8195ea42ba" is_pre_election: true
I20260808 18:23:05.093169 32383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c5255c35ee4e10b88c256de01352bf" candidate_uuid: "0aba19b7cb8a402a822a2fd608e3f697" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bbded853ce94d80b70ff5059443baca" is_pre_election: true
I20260808 18:23:05.093267 32546 raft_consensus.cc:2454] T f9c5255c35ee4e10b88c256de01352bf P b340acac117a4ec8a029ab8195ea42ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0aba19b7cb8a402a822a2fd608e3f697 in term 0.
I20260808 18:23:05.093463 32383 raft_consensus.cc:2454] T f9c5255c35ee4e10b88c256de01352bf P 6bbded853ce94d80b70ff5059443baca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0aba19b7cb8a402a822a2fd608e3f697 in term 0.
I20260808 18:23:05.093762 32161 leader_election.cc:302] T f9c5255c35ee4e10b88c256de01352bf P 0aba19b7cb8a402a822a2fd608e3f697 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0aba19b7cb8a402a822a2fd608e3f697, b340acac117a4ec8a029ab8195ea42ba; no voters:
I20260808 18:23:05.094023 32641 raft_consensus.cc:2781] T f9c5255c35ee4e10b88c256de01352bf P 0aba19b7cb8a402a822a2fd608e3f697 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:23:05.094097 32641 raft_consensus.cc:482] T f9c5255c35ee4e10b88c256de01352bf P 0aba19b7cb8a402a822a2fd608e3f697 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:23:05.094127 32641 raft_consensus.cc:3037] T f9c5255c35ee4e10b88c256de01352bf P 0aba19b7cb8a402a822a2fd608e3f697 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:23:05.095180 32641 raft_consensus.cc:504] T f9c5255c35ee4e10b88c256de01352bf P 0aba19b7cb8a402a822a2fd608e3f697 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } }
I20260808 18:23:05.095646 32641 leader_election.cc:288] T f9c5255c35ee4e10b88c256de01352bf P 0aba19b7cb8a402a822a2fd608e3f697 [CANDIDATE]: Term 1 election: Requested vote from peers b340acac117a4ec8a029ab8195ea42ba (127.23.72.131:41059), 6bbded853ce94d80b70ff5059443baca (127.23.72.130:43777)
I20260808 18:23:05.095961 32546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c5255c35ee4e10b88c256de01352bf" candidate_uuid: "0aba19b7cb8a402a822a2fd608e3f697" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b340acac117a4ec8a029ab8195ea42ba"
I20260808 18:23:05.096074 32383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c5255c35ee4e10b88c256de01352bf" candidate_uuid: "0aba19b7cb8a402a822a2fd608e3f697" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bbded853ce94d80b70ff5059443baca"
I20260808 18:23:05.096084 32546 raft_consensus.cc:3037] T f9c5255c35ee4e10b88c256de01352bf P b340acac117a4ec8a029ab8195ea42ba [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:23:05.096196 32383 raft_consensus.cc:3037] T f9c5255c35ee4e10b88c256de01352bf P 6bbded853ce94d80b70ff5059443baca [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:23:05.097313 32383 raft_consensus.cc:2454] T f9c5255c35ee4e10b88c256de01352bf P 6bbded853ce94d80b70ff5059443baca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0aba19b7cb8a402a822a2fd608e3f697 in term 1.
I20260808 18:23:05.097528 32546 raft_consensus.cc:2454] T f9c5255c35ee4e10b88c256de01352bf P b340acac117a4ec8a029ab8195ea42ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0aba19b7cb8a402a822a2fd608e3f697 in term 1.
I20260808 18:23:05.097661 32161 leader_election.cc:302] T f9c5255c35ee4e10b88c256de01352bf P 0aba19b7cb8a402a822a2fd608e3f697 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0aba19b7cb8a402a822a2fd608e3f697, 6bbded853ce94d80b70ff5059443baca; no voters:
I20260808 18:23:05.097842 32641 raft_consensus.cc:2781] T f9c5255c35ee4e10b88c256de01352bf P 0aba19b7cb8a402a822a2fd608e3f697 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:23:05.098062 32641 raft_consensus.cc:686] T f9c5255c35ee4e10b88c256de01352bf P 0aba19b7cb8a402a822a2fd608e3f697 [term 1 LEADER]: Becoming Leader. State: Replica: 0aba19b7cb8a402a822a2fd608e3f697, State: Running, Role: LEADER
I20260808 18:23:05.098404 32641 consensus_queue.cc:237] T f9c5255c35ee4e10b88c256de01352bf P 0aba19b7cb8a402a822a2fd608e3f697 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } }
I20260808 18:23:05.101619 32051 catalog_manager.cc:5495] T f9c5255c35ee4e10b88c256de01352bf P 0aba19b7cb8a402a822a2fd608e3f697 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0aba19b7cb8a402a822a2fd608e3f697 (127.23.72.129). New cstate: current_term: 1 leader_uuid: "0aba19b7cb8a402a822a2fd608e3f697" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } health_report { overall_health: HEALTHY } } }
I20260808 18:23:05.149842 32630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:05.146282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54358 (local address 127.23.72.129:44001)
0808 18:23:05.146431 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:05.146435 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:05.146450 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:23:05.146584 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:05.146586 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:05.146642 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:05.146733 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:23:05.147017 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:05.147548 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:05.148418 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:05.148628 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:05.148709 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:05.148993 (+ 284us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:23:05.149446 (+ 453us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:23:05.149534 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:05.149613 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 18:23:05.149660 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 18:23:05.156080 32383 raft_consensus.cc:1261] T f9c5255c35ee4e10b88c256de01352bf P 6bbded853ce94d80b70ff5059443baca [term 1 FOLLOWER]: Refusing update from remote peer 0aba19b7cb8a402a822a2fd608e3f697: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:23:05.156108 32546 raft_consensus.cc:1261] T f9c5255c35ee4e10b88c256de01352bf P b340acac117a4ec8a029ab8195ea42ba [term 1 FOLLOWER]: Refusing update from remote peer 0aba19b7cb8a402a822a2fd608e3f697: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:23:05.156804 32641 consensus_queue.cc:1035] T f9c5255c35ee4e10b88c256de01352bf P 0aba19b7cb8a402a822a2fd608e3f697 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:23:05.156996 32646 consensus_queue.cc:1035] T f9c5255c35ee4e10b88c256de01352bf P 0aba19b7cb8a402a822a2fd608e3f697 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:23:05.163938 32649 mvcc.cc:205] Tried to move back new op lower bound from 7316330025591623680 to 7316330025365893120. Current Snapshot: MvccSnapshot[applied={T|T < 7316330025591623680}]
I20260808 18:23:05.165951 32650 mvcc.cc:205] Tried to move back new op lower bound from 7316330025591623680 to 7316330025365893120. Current Snapshot: MvccSnapshot[applied={T|T < 7316330025591623680}]
I20260808 18:23:05.172554 32050 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36944:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:23:05.172700 32050 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36944:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:23:05.175299 32050 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f0f38b8754bb4ca6a2ec3310e69c9c55: Sending DeleteTablet for 3 replicas of tablet f9c5255c35ee4e10b88c256de01352bf
I20260808 18:23:05.176056 32524 tablet_service.cc:1514] Processing DeleteTablet for tablet f9c5255c35ee4e10b88c256de01352bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:23:05 UTC) from {username='oryx', principal='oryx/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:53204
I20260808 18:23:05.176117 32362 tablet_service.cc:1514] Processing DeleteTablet for tablet f9c5255c35ee4e10b88c256de01352bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:23:05 UTC) from {username='oryx', principal='oryx/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:57436
I20260808 18:23:05.176329 32213 tablet_service.cc:1514] Processing DeleteTablet for tablet f9c5255c35ee4e10b88c256de01352bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:23:05 UTC) from {username='oryx', principal='oryx/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:54354
I20260808 18:23:05.176671 32661 tablet_replica.cc:330] T f9c5255c35ee4e10b88c256de01352bf P 6bbded853ce94d80b70ff5059443baca: stopping tablet replica
I20260808 18:23:05.176738 32664 tablet_replica.cc:330] T f9c5255c35ee4e10b88c256de01352bf P 0aba19b7cb8a402a822a2fd608e3f697: stopping tablet replica
I20260808 18:23:05.177050 32664 raft_consensus.cc:2229] T f9c5255c35ee4e10b88c256de01352bf P 0aba19b7cb8a402a822a2fd608e3f697 [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:23:05.177105 32661 raft_consensus.cc:2229] T f9c5255c35ee4e10b88c256de01352bf P 6bbded853ce94d80b70ff5059443baca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:23:05.177315 32050 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36944:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 18:23:05.177461 32664 raft_consensus.cc:2258] T f9c5255c35ee4e10b88c256de01352bf P 0aba19b7cb8a402a822a2fd608e3f697 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 18:23:05.177721 32050 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:23:05.177783 32661 raft_consensus.cc:2258] T f9c5255c35ee4e10b88c256de01352bf P 6bbded853ce94d80b70ff5059443baca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:23:05.178998 32664 ts_tablet_manager.cc:1905] T f9c5255c35ee4e10b88c256de01352bf P 0aba19b7cb8a402a822a2fd608e3f697: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:23:05.179543 32660 tablet_replica.cc:330] T f9c5255c35ee4e10b88c256de01352bf P b340acac117a4ec8a029ab8195ea42ba: stopping tablet replica
I20260808 18:23:05.179679 32661 ts_tablet_manager.cc:1905] T f9c5255c35ee4e10b88c256de01352bf P 6bbded853ce94d80b70ff5059443baca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:23:05.179826 32660 raft_consensus.cc:2229] T f9c5255c35ee4e10b88c256de01352bf P b340acac117a4ec8a029ab8195ea42ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:23:05.180141 32660 raft_consensus.cc:2258] T f9c5255c35ee4e10b88c256de01352bf P b340acac117a4ec8a029ab8195ea42ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:23:05.182822 32661 ts_tablet_manager.cc:1918] T f9c5255c35ee4e10b88c256de01352bf P 6bbded853ce94d80b70ff5059443baca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:23:05.182989 32661 log.cc:1196] T f9c5255c35ee4e10b88c256de01352bf P 6bbded853ce94d80b70ff5059443baca: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-1/wal/wals/f9c5255c35ee4e10b88c256de01352bf
I20260808 18:23:05.183444 32661 ts_tablet_manager.cc:1939] T f9c5255c35ee4e10b88c256de01352bf P 6bbded853ce94d80b70ff5059443baca: Deleting consensus metadata
I20260808 18:23:05.183521 32664 ts_tablet_manager.cc:1918] T f9c5255c35ee4e10b88c256de01352bf P 0aba19b7cb8a402a822a2fd608e3f697: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:23:05.183630 32664 log.cc:1196] T f9c5255c35ee4e10b88c256de01352bf P 0aba19b7cb8a402a822a2fd608e3f697: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-0/wal/wals/f9c5255c35ee4e10b88c256de01352bf
I20260808 18:23:05.184017 32664 ts_tablet_manager.cc:1939] T f9c5255c35ee4e10b88c256de01352bf P 0aba19b7cb8a402a822a2fd608e3f697: Deleting consensus metadata
I20260808 18:23:05.184279 32362 tablet_service.cc:1467] Processing CreateTablet for tablet b9bea728fc344c749cd9ff72059008d6 (DEFAULT_TABLE table=test-table [id=76e73b8109254127bacc3ab8049e0fff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:23:05.184531 32362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9bea728fc344c749cd9ff72059008d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:23:05.186332 32635 tablet_bootstrap.cc:492] T b9bea728fc344c749cd9ff72059008d6 P 6bbded853ce94d80b70ff5059443baca: Bootstrap starting.
I20260808 18:23:05.186545 32524 tablet_service.cc:1467] Processing CreateTablet for tablet b9bea728fc344c749cd9ff72059008d6 (DEFAULT_TABLE table=test-table [id=76e73b8109254127bacc3ab8049e0fff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:23:05.186810 32524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9bea728fc344c749cd9ff72059008d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:23:05.187170 32037 catalog_manager.cc:4841] TS 0aba19b7cb8a402a822a2fd608e3f697 (127.23.72.129:44001): tablet f9c5255c35ee4e10b88c256de01352bf (table test-table [id=bf157dce75c1470986865d4c18cfadfe]) successfully deleted
I20260808 18:23:05.187274 32635 tablet_bootstrap.cc:654] T b9bea728fc344c749cd9ff72059008d6 P 6bbded853ce94d80b70ff5059443baca: Neither blocks nor log segments found. Creating new log.
I20260808 18:23:05.188433 32660 ts_tablet_manager.cc:1905] T f9c5255c35ee4e10b88c256de01352bf P b340acac117a4ec8a029ab8195ea42ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:23:05.189220 32634 tablet_bootstrap.cc:492] T b9bea728fc344c749cd9ff72059008d6 P b340acac117a4ec8a029ab8195ea42ba: Bootstrap starting.
I20260808 18:23:05.189422 32213 tablet_service.cc:1467] Processing CreateTablet for tablet b9bea728fc344c749cd9ff72059008d6 (DEFAULT_TABLE table=test-table [id=76e73b8109254127bacc3ab8049e0fff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:23:05.189662 32213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9bea728fc344c749cd9ff72059008d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:23:05.190155 32634 tablet_bootstrap.cc:654] T b9bea728fc344c749cd9ff72059008d6 P b340acac117a4ec8a029ab8195ea42ba: Neither blocks nor log segments found. Creating new log.
I20260808 18:23:05.191043 32660 ts_tablet_manager.cc:1918] T f9c5255c35ee4e10b88c256de01352bf P b340acac117a4ec8a029ab8195ea42ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:23:05.191141 32660 log.cc:1196] T f9c5255c35ee4e10b88c256de01352bf P b340acac117a4ec8a029ab8195ea42ba: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-2/wal/wals/f9c5255c35ee4e10b88c256de01352bf
I20260808 18:23:05.191449 32634 tablet_bootstrap.cc:492] T b9bea728fc344c749cd9ff72059008d6 P b340acac117a4ec8a029ab8195ea42ba: No bootstrap required, opened a new log
I20260808 18:23:05.191504 32660 ts_tablet_manager.cc:1939] T f9c5255c35ee4e10b88c256de01352bf P b340acac117a4ec8a029ab8195ea42ba: Deleting consensus metadata
I20260808 18:23:05.191524 32634 ts_tablet_manager.cc:1397] T b9bea728fc344c749cd9ff72059008d6 P b340acac117a4ec8a029ab8195ea42ba: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260808 18:23:05.192057 32633 tablet_bootstrap.cc:492] T b9bea728fc344c749cd9ff72059008d6 P 0aba19b7cb8a402a822a2fd608e3f697: Bootstrap starting.
I20260808 18:23:05.191982 32634 raft_consensus.cc:348] T b9bea728fc344c749cd9ff72059008d6 P b340acac117a4ec8a029ab8195ea42ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } }
I20260808 18:23:05.192094 32634 raft_consensus.cc:374] T b9bea728fc344c749cd9ff72059008d6 P b340acac117a4ec8a029ab8195ea42ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:23:05.192128 32634 raft_consensus.cc:729] T b9bea728fc344c749cd9ff72059008d6 P b340acac117a4ec8a029ab8195ea42ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b340acac117a4ec8a029ab8195ea42ba, State: Initialized, Role: FOLLOWER
I20260808 18:23:05.192247 32634 consensus_queue.cc:260] T b9bea728fc344c749cd9ff72059008d6 P b340acac117a4ec8a029ab8195ea42ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } }
I20260808 18:23:05.192528 32634 ts_tablet_manager.cc:1428] T b9bea728fc344c749cd9ff72059008d6 P b340acac117a4ec8a029ab8195ea42ba: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:23:05.193041 32633 tablet_bootstrap.cc:654] T b9bea728fc344c749cd9ff72059008d6 P 0aba19b7cb8a402a822a2fd608e3f697: Neither blocks nor log segments found. Creating new log.
I20260808 18:23:05.193148 32038 catalog_manager.cc:4841] TS b340acac117a4ec8a029ab8195ea42ba (127.23.72.131:41059): tablet f9c5255c35ee4e10b88c256de01352bf (table test-table [id=bf157dce75c1470986865d4c18cfadfe]) successfully deleted
I20260808 18:23:05.193287 32038 catalog_manager.cc:4841] TS 6bbded853ce94d80b70ff5059443baca (127.23.72.130:43777): tablet f9c5255c35ee4e10b88c256de01352bf (table test-table [id=bf157dce75c1470986865d4c18cfadfe]) successfully deleted
I20260808 18:23:05.194509 32635 tablet_bootstrap.cc:492] T b9bea728fc344c749cd9ff72059008d6 P 6bbded853ce94d80b70ff5059443baca: No bootstrap required, opened a new log
I20260808 18:23:05.194602 32635 ts_tablet_manager.cc:1397] T b9bea728fc344c749cd9ff72059008d6 P 6bbded853ce94d80b70ff5059443baca: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.002s
I20260808 18:23:05.195050 32635 raft_consensus.cc:348] T b9bea728fc344c749cd9ff72059008d6 P 6bbded853ce94d80b70ff5059443baca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } }
I20260808 18:23:05.195168 32635 raft_consensus.cc:374] T b9bea728fc344c749cd9ff72059008d6 P 6bbded853ce94d80b70ff5059443baca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:23:05.195211 32635 raft_consensus.cc:729] T b9bea728fc344c749cd9ff72059008d6 P 6bbded853ce94d80b70ff5059443baca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bbded853ce94d80b70ff5059443baca, State: Initialized, Role: FOLLOWER
I20260808 18:23:05.195225 32633 tablet_bootstrap.cc:492] T b9bea728fc344c749cd9ff72059008d6 P 0aba19b7cb8a402a822a2fd608e3f697: No bootstrap required, opened a new log
I20260808 18:23:05.195302 32633 ts_tablet_manager.cc:1397] T b9bea728fc344c749cd9ff72059008d6 P 0aba19b7cb8a402a822a2fd608e3f697: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 18:23:05.195339 32635 consensus_queue.cc:260] T b9bea728fc344c749cd9ff72059008d6 P 6bbded853ce94d80b70ff5059443baca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } }
I20260808 18:23:05.195640 32635 ts_tablet_manager.cc:1428] T b9bea728fc344c749cd9ff72059008d6 P 6bbded853ce94d80b70ff5059443baca: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 18:23:05.195782 32633 raft_consensus.cc:348] T b9bea728fc344c749cd9ff72059008d6 P 0aba19b7cb8a402a822a2fd608e3f697 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } }
I20260808 18:23:05.195904 32633 raft_consensus.cc:374] T b9bea728fc344c749cd9ff72059008d6 P 0aba19b7cb8a402a822a2fd608e3f697 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:23:05.195940 32633 raft_consensus.cc:729] T b9bea728fc344c749cd9ff72059008d6 P 0aba19b7cb8a402a822a2fd608e3f697 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0aba19b7cb8a402a822a2fd608e3f697, State: Initialized, Role: FOLLOWER
I20260808 18:23:05.196069 32633 consensus_queue.cc:260] T b9bea728fc344c749cd9ff72059008d6 P 0aba19b7cb8a402a822a2fd608e3f697 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } }
I20260808 18:23:05.196349 32633 ts_tablet_manager.cc:1428] T b9bea728fc344c749cd9ff72059008d6 P 0aba19b7cb8a402a822a2fd608e3f697: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:23:05.309365 32640 raft_consensus.cc:482] T b9bea728fc344c749cd9ff72059008d6 P b340acac117a4ec8a029ab8195ea42ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:23:05.309561 32640 raft_consensus.cc:504] T b9bea728fc344c749cd9ff72059008d6 P b340acac117a4ec8a029ab8195ea42ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } }
I20260808 18:23:05.311019 32640 leader_election.cc:288] T b9bea728fc344c749cd9ff72059008d6 P b340acac117a4ec8a029ab8195ea42ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0aba19b7cb8a402a822a2fd608e3f697 (127.23.72.129:44001), 6bbded853ce94d80b70ff5059443baca (127.23.72.130:43777)
I20260808 18:23:05.315562 32632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:05.311526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:51701 (local address 127.23.72.130:43777)
0808 18:23:05.311716 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:05.311723 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:05.311983 (+ 260us) server_negotiation.cc:407] Connection header received
0808 18:23:05.312129 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:05.312134 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:05.312183 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:05.312248 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:23:05.312574 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:05.313314 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:05.314385 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:05.314829 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:05.314943 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:05.315003 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:05.315324 (+ 321us) server_negotiation.cc:299] Negotiation successful
0808 18:23:05.315406 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260808 18:23:05.316021 32674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:05.311248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.130:43777 (local address 127.23.72.131:51701)
0808 18:23:05.311902 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:05.311914 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:05.312020 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:05.312298 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:05.312302 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:05.312325 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:23:05.312439 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:05.312447 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:05.313405 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:05.313410 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:05.314252 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:05.314261 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:05.315068 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:05.315072 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:05.315180 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:05.315205 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 18:23:05.315750 (+ 545us) client_negotiation.cc:240] Negotiation successful
0808 18:23:05.315833 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"mutex_wait_us":237,"thread_start_us":50,"threads_started":1}
I20260808 18:23:05.316480 32383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9bea728fc344c749cd9ff72059008d6" candidate_uuid: "b340acac117a4ec8a029ab8195ea42ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bbded853ce94d80b70ff5059443baca" is_pre_election: true
I20260808 18:23:05.316643 32383 raft_consensus.cc:2454] T b9bea728fc344c749cd9ff72059008d6 P 6bbded853ce94d80b70ff5059443baca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b340acac117a4ec8a029ab8195ea42ba in term 0.
I20260808 18:23:05.316705 32630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:05.311719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:41729 (local address 127.23.72.129:44001)
0808 18:23:05.311876 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:05.311881 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:05.311900 (+ 19us) server_negotiation.cc:407] Connection header received
0808 18:23:05.311962 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:05.311966 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:05.312022 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:05.312128 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:23:05.312433 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:05.313226 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:05.314974 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:05.315698 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:05.315808 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:05.315862 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:05.316482 (+ 620us) server_negotiation.cc:299] Negotiation successful
0808 18:23:05.316573 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 18:23:05.317042 32469 leader_election.cc:302] T b9bea728fc344c749cd9ff72059008d6 P b340acac117a4ec8a029ab8195ea42ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bbded853ce94d80b70ff5059443baca, b340acac117a4ec8a029ab8195ea42ba; no voters:
I20260808 18:23:05.317203 32672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:05.311248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:44001 (local address 127.23.72.131:41729)
0808 18:23:05.311658 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:05.311673 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:05.311787 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:05.312150 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:05.312153 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:05.312168 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:23:05.312305 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:05.312314 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:05.313377 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:05.313381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:05.314798 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:05.314811 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:05.316154 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:05.316159 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:05.316269 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:05.316333 (+ 64us) client_negotiation.cc:769] Sending connection context
0808 18:23:05.316934 (+ 601us) client_negotiation.cc:240] Negotiation successful
0808 18:23:05.317021 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"mutex_wait_us":54,"thread_start_us":98,"threads_started":1}
I20260808 18:23:05.317344 32640 raft_consensus.cc:2781] T b9bea728fc344c749cd9ff72059008d6 P b340acac117a4ec8a029ab8195ea42ba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:23:05.317414 32640 raft_consensus.cc:482] T b9bea728fc344c749cd9ff72059008d6 P b340acac117a4ec8a029ab8195ea42ba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:23:05.317464 32640 raft_consensus.cc:3037] T b9bea728fc344c749cd9ff72059008d6 P b340acac117a4ec8a029ab8195ea42ba [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:23:05.317883 32236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9bea728fc344c749cd9ff72059008d6" candidate_uuid: "b340acac117a4ec8a029ab8195ea42ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0aba19b7cb8a402a822a2fd608e3f697" is_pre_election: true
I20260808 18:23:05.318145 32236 raft_consensus.cc:2454] T b9bea728fc344c749cd9ff72059008d6 P 0aba19b7cb8a402a822a2fd608e3f697 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b340acac117a4ec8a029ab8195ea42ba in term 0.
I20260808 18:23:05.318858 32640 raft_consensus.cc:504] T b9bea728fc344c749cd9ff72059008d6 P b340acac117a4ec8a029ab8195ea42ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } }
I20260808 18:23:05.319273 32640 leader_election.cc:288] T b9bea728fc344c749cd9ff72059008d6 P b340acac117a4ec8a029ab8195ea42ba [CANDIDATE]: Term 1 election: Requested vote from peers 0aba19b7cb8a402a822a2fd608e3f697 (127.23.72.129:44001), 6bbded853ce94d80b70ff5059443baca (127.23.72.130:43777)
I20260808 18:23:05.319653 32236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9bea728fc344c749cd9ff72059008d6" candidate_uuid: "b340acac117a4ec8a029ab8195ea42ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0aba19b7cb8a402a822a2fd608e3f697"
I20260808 18:23:05.319643 32383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9bea728fc344c749cd9ff72059008d6" candidate_uuid: "b340acac117a4ec8a029ab8195ea42ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bbded853ce94d80b70ff5059443baca"
I20260808 18:23:05.319759 32236 raft_consensus.cc:3037] T b9bea728fc344c749cd9ff72059008d6 P 0aba19b7cb8a402a822a2fd608e3f697 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:23:05.319761 32383 raft_consensus.cc:3037] T b9bea728fc344c749cd9ff72059008d6 P 6bbded853ce94d80b70ff5059443baca [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:23:05.320915 32383 raft_consensus.cc:2454] T b9bea728fc344c749cd9ff72059008d6 P 6bbded853ce94d80b70ff5059443baca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b340acac117a4ec8a029ab8195ea42ba in term 1.
I20260808 18:23:05.320915 32236 raft_consensus.cc:2454] T b9bea728fc344c749cd9ff72059008d6 P 0aba19b7cb8a402a822a2fd608e3f697 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b340acac117a4ec8a029ab8195ea42ba in term 1.
I20260808 18:23:05.321219 32469 leader_election.cc:302] T b9bea728fc344c749cd9ff72059008d6 P b340acac117a4ec8a029ab8195ea42ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bbded853ce94d80b70ff5059443baca, b340acac117a4ec8a029ab8195ea42ba; no voters:
I20260808 18:23:05.321450 32640 raft_consensus.cc:2781] T b9bea728fc344c749cd9ff72059008d6 P b340acac117a4ec8a029ab8195ea42ba [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:23:05.321683 32640 raft_consensus.cc:686] T b9bea728fc344c749cd9ff72059008d6 P b340acac117a4ec8a029ab8195ea42ba [term 1 LEADER]: Becoming Leader. State: Replica: b340acac117a4ec8a029ab8195ea42ba, State: Running, Role: LEADER
I20260808 18:23:05.321931 32640 consensus_queue.cc:237] T b9bea728fc344c749cd9ff72059008d6 P b340acac117a4ec8a029ab8195ea42ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } }
I20260808 18:23:05.324414 32052 catalog_manager.cc:5495] T b9bea728fc344c749cd9ff72059008d6 P b340acac117a4ec8a029ab8195ea42ba reported cstate change: term changed from 0 to 1, leader changed from <none> to b340acac117a4ec8a029ab8195ea42ba (127.23.72.131). New cstate: current_term: 1 leader_uuid: "b340acac117a4ec8a029ab8195ea42ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } health_report { overall_health: HEALTHY } } }
Aug 08 18:23:05 dist-test-slave-5k54 krb5kdc[32004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213383, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.72.190@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM
I20260808 18:23:05.374387 32682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:05.365130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:40515 (local address 127.0.0.1:36960)
0808 18:23:05.365467 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:05.365482 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:05.365577 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:05.365880 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:05.365883 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:05.366269 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:05.366366 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:05.366373 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:05.367159 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:05.367162 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:05.367867 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:05.367877 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:05.368508 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:05.368512 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:05.368582 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:05.369158 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:05.369169 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:05.370699 (+ 1530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:23:05.372649 (+ 1950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:23:05.372658 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:23:05.372662 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:23:05.372898 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:23:05.373325 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:23:05.373329 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:23:05.373332 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:23:05.373426 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:23:05.373828 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:23:05.373834 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:23:05.373937 (+ 103us) client_negotiation.cc:769] Sending connection context
0808 18:23:05.374033 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 18:23:05.374244 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":99,"threads_started":1}
I20260808 18:23:05.374544 32683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:05.365192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36960 (local address 127.23.72.190:40515)
0808 18:23:05.365525 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:05.365537 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:05.365553 (+ 16us) server_negotiation.cc:407] Connection header received
0808 18:23:05.365709 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:05.365711 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:05.365755 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:05.365839 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:05.366489 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:05.367053 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:05.367987 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:05.368234 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:05.368335 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:05.370874 (+ 2539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:23:05.370877 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:23:05.370879 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:23:05.370900 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:23:05.372544 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:23:05.373007 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:23:05.373010 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:23:05.373012 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:23:05.373109 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:23:05.373523 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:23:05.373526 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:23:05.373527 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:23:05.373724 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:23:05.373817 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:05.374130 (+ 313us) server_negotiation.cc:299] Negotiation successful
0808 18:23:05.374315 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":260,"thread_start_us":147,"threads_started":1}
I20260808 18:23:05.378499 32052 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:36960:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 18:23:05.379269 32052 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:23:05.384203 32362 tablet_service.cc:1467] Processing CreateTablet for tablet 7da5cef9ef484ce5bacae6bc4494e810 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=70da3b323a7d4c019b75aabd4a1e7aa4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 18:23:05.384195 32213 tablet_service.cc:1467] Processing CreateTablet for tablet 7da5cef9ef484ce5bacae6bc4494e810 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=70da3b323a7d4c019b75aabd4a1e7aa4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 18:23:05.384284 32524 tablet_service.cc:1467] Processing CreateTablet for tablet 7da5cef9ef484ce5bacae6bc4494e810 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=70da3b323a7d4c019b75aabd4a1e7aa4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 18:23:05.384604 32362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7da5cef9ef484ce5bacae6bc4494e810. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:23:05.384604 32213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7da5cef9ef484ce5bacae6bc4494e810. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:23:05.384701 32524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7da5cef9ef484ce5bacae6bc4494e810. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:23:05.387665 32635 tablet_bootstrap.cc:492] T 7da5cef9ef484ce5bacae6bc4494e810 P 6bbded853ce94d80b70ff5059443baca: Bootstrap starting.
I20260808 18:23:05.387684 32634 tablet_bootstrap.cc:492] T 7da5cef9ef484ce5bacae6bc4494e810 P b340acac117a4ec8a029ab8195ea42ba: Bootstrap starting.
I20260808 18:23:05.387985 32633 tablet_bootstrap.cc:492] T 7da5cef9ef484ce5bacae6bc4494e810 P 0aba19b7cb8a402a822a2fd608e3f697: Bootstrap starting.
I20260808 18:23:05.388615 32635 tablet_bootstrap.cc:654] T 7da5cef9ef484ce5bacae6bc4494e810 P 6bbded853ce94d80b70ff5059443baca: Neither blocks nor log segments found. Creating new log.
I20260808 18:23:05.388686 32634 tablet_bootstrap.cc:654] T 7da5cef9ef484ce5bacae6bc4494e810 P b340acac117a4ec8a029ab8195ea42ba: Neither blocks nor log segments found. Creating new log.
I20260808 18:23:05.388794 32633 tablet_bootstrap.cc:654] T 7da5cef9ef484ce5bacae6bc4494e810 P 0aba19b7cb8a402a822a2fd608e3f697: Neither blocks nor log segments found. Creating new log.
I20260808 18:23:05.389781 32635 tablet_bootstrap.cc:492] T 7da5cef9ef484ce5bacae6bc4494e810 P 6bbded853ce94d80b70ff5059443baca: No bootstrap required, opened a new log
I20260808 18:23:05.389899 32635 ts_tablet_manager.cc:1397] T 7da5cef9ef484ce5bacae6bc4494e810 P 6bbded853ce94d80b70ff5059443baca: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 18:23:05.390226 32634 tablet_bootstrap.cc:492] T 7da5cef9ef484ce5bacae6bc4494e810 P b340acac117a4ec8a029ab8195ea42ba: No bootstrap required, opened a new log
I20260808 18:23:05.390314 32634 ts_tablet_manager.cc:1397] T 7da5cef9ef484ce5bacae6bc4494e810 P b340acac117a4ec8a029ab8195ea42ba: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 18:23:05.390369 32635 raft_consensus.cc:348] T 7da5cef9ef484ce5bacae6bc4494e810 P 6bbded853ce94d80b70ff5059443baca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } }
I20260808 18:23:05.390496 32635 raft_consensus.cc:374] T 7da5cef9ef484ce5bacae6bc4494e810 P 6bbded853ce94d80b70ff5059443baca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:23:05.390528 32635 raft_consensus.cc:729] T 7da5cef9ef484ce5bacae6bc4494e810 P 6bbded853ce94d80b70ff5059443baca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bbded853ce94d80b70ff5059443baca, State: Initialized, Role: FOLLOWER
I20260808 18:23:05.390659 32635 consensus_queue.cc:260] T 7da5cef9ef484ce5bacae6bc4494e810 P 6bbded853ce94d80b70ff5059443baca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } }
I20260808 18:23:05.390779 32634 raft_consensus.cc:348] T 7da5cef9ef484ce5bacae6bc4494e810 P b340acac117a4ec8a029ab8195ea42ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } }
I20260808 18:23:05.390889 32634 raft_consensus.cc:374] T 7da5cef9ef484ce5bacae6bc4494e810 P b340acac117a4ec8a029ab8195ea42ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:23:05.390923 32634 raft_consensus.cc:729] T 7da5cef9ef484ce5bacae6bc4494e810 P b340acac117a4ec8a029ab8195ea42ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b340acac117a4ec8a029ab8195ea42ba, State: Initialized, Role: FOLLOWER
I20260808 18:23:05.390946 32643 tablet_replica.cc:439] T 7da5cef9ef484ce5bacae6bc4494e810 P 6bbded853ce94d80b70ff5059443baca: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } } }
I20260808 18:23:05.391098 32643 tablet_replica.cc:442] T 7da5cef9ef484ce5bacae6bc4494e810 P 6bbded853ce94d80b70ff5059443baca: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:23:05.391040 32634 consensus_queue.cc:260] T 7da5cef9ef484ce5bacae6bc4494e810 P b340acac117a4ec8a029ab8195ea42ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } }
I20260808 18:23:05.390962 32635 ts_tablet_manager.cc:1428] T 7da5cef9ef484ce5bacae6bc4494e810 P 6bbded853ce94d80b70ff5059443baca: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 18:23:05.391379 32634 ts_tablet_manager.cc:1428] T 7da5cef9ef484ce5bacae6bc4494e810 P b340acac117a4ec8a029ab8195ea42ba: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:23:05.391336 32640 tablet_replica.cc:439] T 7da5cef9ef484ce5bacae6bc4494e810 P b340acac117a4ec8a029ab8195ea42ba: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } } }
I20260808 18:23:05.391481 32640 tablet_replica.cc:442] T 7da5cef9ef484ce5bacae6bc4494e810 P b340acac117a4ec8a029ab8195ea42ba: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:23:05.391893 32633 tablet_bootstrap.cc:492] T 7da5cef9ef484ce5bacae6bc4494e810 P 0aba19b7cb8a402a822a2fd608e3f697: No bootstrap required, opened a new log
I20260808 18:23:05.391968 32633 ts_tablet_manager.cc:1397] T 7da5cef9ef484ce5bacae6bc4494e810 P 0aba19b7cb8a402a822a2fd608e3f697: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260808 18:23:05.392381 32633 raft_consensus.cc:348] T 7da5cef9ef484ce5bacae6bc4494e810 P 0aba19b7cb8a402a822a2fd608e3f697 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } }
I20260808 18:23:05.392495 32633 raft_consensus.cc:374] T 7da5cef9ef484ce5bacae6bc4494e810 P 0aba19b7cb8a402a822a2fd608e3f697 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:23:05.392519 32633 raft_consensus.cc:729] T 7da5cef9ef484ce5bacae6bc4494e810 P 0aba19b7cb8a402a822a2fd608e3f697 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0aba19b7cb8a402a822a2fd608e3f697, State: Initialized, Role: FOLLOWER
I20260808 18:23:05.392640 32633 consensus_queue.cc:260] T 7da5cef9ef484ce5bacae6bc4494e810 P 0aba19b7cb8a402a822a2fd608e3f697 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } }
I20260808 18:23:05.392910 32633 ts_tablet_manager.cc:1428] T 7da5cef9ef484ce5bacae6bc4494e810 P 0aba19b7cb8a402a822a2fd608e3f697: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 18:23:05.393117 32646 tablet_replica.cc:439] T 7da5cef9ef484ce5bacae6bc4494e810 P 0aba19b7cb8a402a822a2fd608e3f697: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } } }
I20260808 18:23:05.393260 32646 tablet_replica.cc:442] T 7da5cef9ef484ce5bacae6bc4494e810 P 0aba19b7cb8a402a822a2fd608e3f697: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:23:05.418659 32646 raft_consensus.cc:482] T 7da5cef9ef484ce5bacae6bc4494e810 P 0aba19b7cb8a402a822a2fd608e3f697 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:23:05.418782 32646 raft_consensus.cc:504] T 7da5cef9ef484ce5bacae6bc4494e810 P 0aba19b7cb8a402a822a2fd608e3f697 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } }
I20260808 18:23:05.419250 32646 leader_election.cc:288] T 7da5cef9ef484ce5bacae6bc4494e810 P 0aba19b7cb8a402a822a2fd608e3f697 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6bbded853ce94d80b70ff5059443baca (127.23.72.130:43777), b340acac117a4ec8a029ab8195ea42ba (127.23.72.131:41059)
I20260808 18:23:05.419659 32383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7da5cef9ef484ce5bacae6bc4494e810" candidate_uuid: "0aba19b7cb8a402a822a2fd608e3f697" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bbded853ce94d80b70ff5059443baca" is_pre_election: true
I20260808 18:23:05.419817 32383 raft_consensus.cc:2454] T 7da5cef9ef484ce5bacae6bc4494e810 P 6bbded853ce94d80b70ff5059443baca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0aba19b7cb8a402a822a2fd608e3f697 in term 0.
I20260808 18:23:05.419755 32546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7da5cef9ef484ce5bacae6bc4494e810" candidate_uuid: "0aba19b7cb8a402a822a2fd608e3f697" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b340acac117a4ec8a029ab8195ea42ba" is_pre_election: true
I20260808 18:23:05.419943 32546 raft_consensus.cc:2454] T 7da5cef9ef484ce5bacae6bc4494e810 P b340acac117a4ec8a029ab8195ea42ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0aba19b7cb8a402a822a2fd608e3f697 in term 0.
I20260808 18:23:05.420135 32161 leader_election.cc:302] T 7da5cef9ef484ce5bacae6bc4494e810 P 0aba19b7cb8a402a822a2fd608e3f697 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0aba19b7cb8a402a822a2fd608e3f697, 6bbded853ce94d80b70ff5059443baca; no voters:
I20260808 18:23:05.420333 32646 raft_consensus.cc:2781] T 7da5cef9ef484ce5bacae6bc4494e810 P 0aba19b7cb8a402a822a2fd608e3f697 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:23:05.420392 32646 raft_consensus.cc:482] T 7da5cef9ef484ce5bacae6bc4494e810 P 0aba19b7cb8a402a822a2fd608e3f697 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:23:05.420439 32646 raft_consensus.cc:3037] T 7da5cef9ef484ce5bacae6bc4494e810 P 0aba19b7cb8a402a822a2fd608e3f697 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:23:05.421499 32646 raft_consensus.cc:504] T 7da5cef9ef484ce5bacae6bc4494e810 P 0aba19b7cb8a402a822a2fd608e3f697 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } }
I20260808 18:23:05.421764 32646 leader_election.cc:288] T 7da5cef9ef484ce5bacae6bc4494e810 P 0aba19b7cb8a402a822a2fd608e3f697 [CANDIDATE]: Term 1 election: Requested vote from peers 6bbded853ce94d80b70ff5059443baca (127.23.72.130:43777), b340acac117a4ec8a029ab8195ea42ba (127.23.72.131:41059)
I20260808 18:23:05.422098 32383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7da5cef9ef484ce5bacae6bc4494e810" candidate_uuid: "0aba19b7cb8a402a822a2fd608e3f697" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bbded853ce94d80b70ff5059443baca"
I20260808 18:23:05.422200 32383 raft_consensus.cc:3037] T 7da5cef9ef484ce5bacae6bc4494e810 P 6bbded853ce94d80b70ff5059443baca [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:23:05.422298 32546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7da5cef9ef484ce5bacae6bc4494e810" candidate_uuid: "0aba19b7cb8a402a822a2fd608e3f697" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b340acac117a4ec8a029ab8195ea42ba"
I20260808 18:23:05.422459 32546 raft_consensus.cc:3037] T 7da5cef9ef484ce5bacae6bc4494e810 P b340acac117a4ec8a029ab8195ea42ba [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:23:05.423028 32383 raft_consensus.cc:2454] T 7da5cef9ef484ce5bacae6bc4494e810 P 6bbded853ce94d80b70ff5059443baca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0aba19b7cb8a402a822a2fd608e3f697 in term 1.
I20260808 18:23:05.423318 32161 leader_election.cc:302] T 7da5cef9ef484ce5bacae6bc4494e810 P 0aba19b7cb8a402a822a2fd608e3f697 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0aba19b7cb8a402a822a2fd608e3f697, 6bbded853ce94d80b70ff5059443baca; no voters:
I20260808 18:23:05.423552 32646 raft_consensus.cc:2781] T 7da5cef9ef484ce5bacae6bc4494e810 P 0aba19b7cb8a402a822a2fd608e3f697 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:23:05.423583 32546 raft_consensus.cc:2454] T 7da5cef9ef484ce5bacae6bc4494e810 P b340acac117a4ec8a029ab8195ea42ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0aba19b7cb8a402a822a2fd608e3f697 in term 1.
I20260808 18:23:05.423671 32646 raft_consensus.cc:686] T 7da5cef9ef484ce5bacae6bc4494e810 P 0aba19b7cb8a402a822a2fd608e3f697 [term 1 LEADER]: Becoming Leader. State: Replica: 0aba19b7cb8a402a822a2fd608e3f697, State: Running, Role: LEADER
I20260808 18:23:05.423795 32646 consensus_queue.cc:237] T 7da5cef9ef484ce5bacae6bc4494e810 P 0aba19b7cb8a402a822a2fd608e3f697 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } }
I20260808 18:23:05.424484 32655 tablet_replica.cc:439] T 7da5cef9ef484ce5bacae6bc4494e810 P 0aba19b7cb8a402a822a2fd608e3f697: TxnStatusTablet state changed. Reason: New leader 0aba19b7cb8a402a822a2fd608e3f697. Latest consensus state: current_term: 1 leader_uuid: "0aba19b7cb8a402a822a2fd608e3f697" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } } }
I20260808 18:23:05.424604 32655 tablet_replica.cc:442] T 7da5cef9ef484ce5bacae6bc4494e810 P 0aba19b7cb8a402a822a2fd608e3f697: This TxnStatusTablet replica's current role is: LEADER
I20260808 18:23:05.424927 32691 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 18:23:05.425107 32691 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 18:23:05.425539 32051 catalog_manager.cc:5495] T 7da5cef9ef484ce5bacae6bc4494e810 P 0aba19b7cb8a402a822a2fd608e3f697 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0aba19b7cb8a402a822a2fd608e3f697 (127.23.72.129). New cstate: current_term: 1 leader_uuid: "0aba19b7cb8a402a822a2fd608e3f697" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } health_report { overall_health: UNKNOWN } } }
I20260808 18:23:05.438544 32630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:05.434881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54372 (local address 127.23.72.129:44001)
0808 18:23:05.435014 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:05.435018 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:05.435033 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:23:05.435203 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:05.435205 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:05.435250 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:05.435341 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:23:05.435648 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:05.436190 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:05.437419 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:05.437682 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:05.437773 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:05.438114 (+ 341us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:23:05.438188 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:23:05.438265 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:05.438381 (+ 116us) server_negotiation.cc:299] Negotiation successful
0808 18:23:05.438431 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 18:23:05.438598 32682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:05.434725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:44001 (local address 127.0.0.1:54372)
0808 18:23:05.434969 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:05.434985 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:05.435082 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:05.435390 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:05.435393 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:05.435401 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 18:23:05.435539 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:05.435545 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:05.436302 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:05.436305 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:05.437285 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:05.437297 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:05.437794 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:05.437798 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:05.437898 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:05.437953 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 18:23:05.438279 (+ 326us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 18:23:05.438294 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 18:23:05.438360 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 18:23:05.438456 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142}
I20260808 18:23:05.441112 32546 raft_consensus.cc:1261] T 7da5cef9ef484ce5bacae6bc4494e810 P b340acac117a4ec8a029ab8195ea42ba [term 1 FOLLOWER]: Refusing update from remote peer 0aba19b7cb8a402a822a2fd608e3f697: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:23:05.441191 32383 raft_consensus.cc:1261] T 7da5cef9ef484ce5bacae6bc4494e810 P 6bbded853ce94d80b70ff5059443baca [term 1 FOLLOWER]: Refusing update from remote peer 0aba19b7cb8a402a822a2fd608e3f697: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:23:05.441547 32646 consensus_queue.cc:1035] T 7da5cef9ef484ce5bacae6bc4494e810 P 0aba19b7cb8a402a822a2fd608e3f697 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:23:05.441720 32655 consensus_queue.cc:1035] T 7da5cef9ef484ce5bacae6bc4494e810 P 0aba19b7cb8a402a822a2fd608e3f697 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:23:05.443143 32643 tablet_replica.cc:439] T 7da5cef9ef484ce5bacae6bc4494e810 P 6bbded853ce94d80b70ff5059443baca: TxnStatusTablet state changed. Reason: New leader 0aba19b7cb8a402a822a2fd608e3f697. Latest consensus state: current_term: 1 leader_uuid: "0aba19b7cb8a402a822a2fd608e3f697" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } } }
I20260808 18:23:05.443269 32643 tablet_replica.cc:442] T 7da5cef9ef484ce5bacae6bc4494e810 P 6bbded853ce94d80b70ff5059443baca: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:23:05.443216 32640 tablet_replica.cc:439] T 7da5cef9ef484ce5bacae6bc4494e810 P b340acac117a4ec8a029ab8195ea42ba: TxnStatusTablet state changed. Reason: New leader 0aba19b7cb8a402a822a2fd608e3f697. Latest consensus state: current_term: 1 leader_uuid: "0aba19b7cb8a402a822a2fd608e3f697" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } } }
I20260808 18:23:05.443378 32640 tablet_replica.cc:442] T 7da5cef9ef484ce5bacae6bc4494e810 P b340acac117a4ec8a029ab8195ea42ba: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:23:05.445253 32655 tablet_replica.cc:439] T 7da5cef9ef484ce5bacae6bc4494e810 P 0aba19b7cb8a402a822a2fd608e3f697: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0aba19b7cb8a402a822a2fd608e3f697" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } } }
I20260808 18:23:05.445358 32655 tablet_replica.cc:442] T 7da5cef9ef484ce5bacae6bc4494e810 P 0aba19b7cb8a402a822a2fd608e3f697: This TxnStatusTablet replica's current role is: LEADER
I20260808 18:23:05.445895 32643 tablet_replica.cc:439] T 7da5cef9ef484ce5bacae6bc4494e810 P 6bbded853ce94d80b70ff5059443baca: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0aba19b7cb8a402a822a2fd608e3f697" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } } }
I20260808 18:23:05.445992 32643 tablet_replica.cc:442] T 7da5cef9ef484ce5bacae6bc4494e810 P 6bbded853ce94d80b70ff5059443baca: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:23:05.449091 32646 tablet_replica.cc:439] T 7da5cef9ef484ce5bacae6bc4494e810 P 0aba19b7cb8a402a822a2fd608e3f697: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0aba19b7cb8a402a822a2fd608e3f697" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } } }
I20260808 18:23:05.449201 32646 tablet_replica.cc:442] T 7da5cef9ef484ce5bacae6bc4494e810 P 0aba19b7cb8a402a822a2fd608e3f697: This TxnStatusTablet replica's current role is: LEADER
I20260808 18:23:05.450076 32640 tablet_replica.cc:439] T 7da5cef9ef484ce5bacae6bc4494e810 P b340acac117a4ec8a029ab8195ea42ba: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0aba19b7cb8a402a822a2fd608e3f697" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } } }
I20260808 18:23:05.450189 32640 tablet_replica.cc:442] T 7da5cef9ef484ce5bacae6bc4494e810 P b340acac117a4ec8a029ab8195ea42ba: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:23:05.450269 32651 mvcc.cc:205] Tried to move back new op lower bound from 7316330026762702848 to 7316330026697228288. Current Snapshot: MvccSnapshot[applied={T|T < 7316330026762702848}]
I20260808 18:23:05.456493 32627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:05.452448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53216 (local address 127.23.72.131:41059)
0808 18:23:05.452543 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:05.452547 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:05.452559 (+ 12us) server_negotiation.cc:407] Connection header received
0808 18:23:05.452617 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:05.452619 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:05.452664 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:05.452747 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:23:05.453007 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:05.453484 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:05.454686 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:05.454992 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:05.455268 (+ 276us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:05.455598 (+ 330us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:23:05.456097 (+ 499us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:23:05.456201 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:05.456302 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 18:23:05.456353 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 18:23:05.467242 32630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:05.463750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:60827 (local address 127.23.72.129:44001)
0808 18:23:05.463891 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:05.463895 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:05.464141 (+ 246us) server_negotiation.cc:407] Connection header received
0808 18:23:05.464296 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:05.464300 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:05.464357 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:05.464439 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:23:05.464687 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:05.465202 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:05.466083 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:05.466332 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:05.466449 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:05.466751 (+ 302us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:23:05.466833 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:23:05.466914 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:05.467054 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 18:23:05.467114 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 18:23:05.468078 32698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:05.463652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:44001 (local address 127.23.72.131:60827)
0808 18:23:05.464047 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:05.464059 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:05.464186 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:05.464451 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:05.464454 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:05.464462 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 18:23:05.464575 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:05.464587 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:05.465326 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:05.465330 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:05.465959 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:05.465967 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:05.466466 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:05.466470 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:05.466572 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:05.466613 (+ 41us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 18:23:05.466922 (+ 309us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 18:23:05.466931 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 18:23:05.467000 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 18:23:05.467080 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":125,"threads_started":1}
I20260808 18:23:05.475132 32236 raft_consensus.cc:1261] T b9bea728fc344c749cd9ff72059008d6 P 0aba19b7cb8a402a822a2fd608e3f697 [term 1 FOLLOWER]: Refusing update from remote peer b340acac117a4ec8a029ab8195ea42ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:23:05.475605 32383 raft_consensus.cc:1261] T b9bea728fc344c749cd9ff72059008d6 P 6bbded853ce94d80b70ff5059443baca [term 1 FOLLOWER]: Refusing update from remote peer b340acac117a4ec8a029ab8195ea42ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:23:05.475867 32675 consensus_queue.cc:1035] T b9bea728fc344c749cd9ff72059008d6 P b340acac117a4ec8a029ab8195ea42ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:23:05.476038 32640 consensus_queue.cc:1035] T b9bea728fc344c749cd9ff72059008d6 P b340acac117a4ec8a029ab8195ea42ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:23:05.512557 32627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:05.508496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:45771 (local address 127.23.72.131:41059)
0808 18:23:05.508613 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:05.508618 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:05.508927 (+ 309us) server_negotiation.cc:407] Connection header received
0808 18:23:05.509107 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:05.509110 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:05.509161 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:05.509205 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:23:05.509598 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:05.510350 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:05.511378 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:05.511735 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:05.511842 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:05.512104 (+ 262us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:23:05.512185 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:23:05.512264 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:05.512366 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 18:23:05.512420 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 18:23:05.512596 32702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:05.508375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.131:41059 (local address 127.23.72.129:45771)
0808 18:23:05.508845 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:05.508860 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:05.509001 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:05.509361 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:05.509364 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:05.509371 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 18:23:05.509472 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:05.509477 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:05.510474 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:05.510478 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:05.511253 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:05.511263 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:05.511861 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:05.511864 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:05.511930 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:05.511977 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 18:23:05.512268 (+ 291us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 18:23:05.512273 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 18:23:05.512337 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 18:23:05.512432 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":132,"threads_started":1}
I20260808 18:23:05.542495 32051 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36944:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:23:05.542678 32051 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36944:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:23:05.544991 32051 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f0f38b8754bb4ca6a2ec3310e69c9c55: Sending DeleteTablet for 3 replicas of tablet b9bea728fc344c749cd9ff72059008d6
I20260808 18:23:05.545521 32213 tablet_service.cc:1514] Processing DeleteTablet for tablet b9bea728fc344c749cd9ff72059008d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:23:05 UTC) from {username='oryx', principal='oryx/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:54354
I20260808 18:23:05.545704 32664 tablet_replica.cc:330] T b9bea728fc344c749cd9ff72059008d6 P 0aba19b7cb8a402a822a2fd608e3f697: stopping tablet replica
I20260808 18:23:05.545737 32524 tablet_service.cc:1514] Processing DeleteTablet for tablet b9bea728fc344c749cd9ff72059008d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:23:05 UTC) from {username='oryx', principal='oryx/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:53204
I20260808 18:23:05.545823 32664 raft_consensus.cc:2229] T b9bea728fc344c749cd9ff72059008d6 P 0aba19b7cb8a402a822a2fd608e3f697 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:23:05.545938 32664 raft_consensus.cc:2258] T b9bea728fc344c749cd9ff72059008d6 P 0aba19b7cb8a402a822a2fd608e3f697 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:23:05.546109 32660 tablet_replica.cc:330] T b9bea728fc344c749cd9ff72059008d6 P b340acac117a4ec8a029ab8195ea42ba: stopping tablet replica
I20260808 18:23:05.546090 32362 tablet_service.cc:1514] Processing DeleteTablet for tablet b9bea728fc344c749cd9ff72059008d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:23:05 UTC) from {username='oryx', principal='oryx/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:57436
I20260808 18:23:05.546223 32660 raft_consensus.cc:2229] T b9bea728fc344c749cd9ff72059008d6 P b340acac117a4ec8a029ab8195ea42ba [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:23:05.546520 32660 raft_consensus.cc:2258] T b9bea728fc344c749cd9ff72059008d6 P b340acac117a4ec8a029ab8195ea42ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:23:05.546805 32661 tablet_replica.cc:330] T b9bea728fc344c749cd9ff72059008d6 P 6bbded853ce94d80b70ff5059443baca: stopping tablet replica
I20260808 18:23:05.546917 32661 raft_consensus.cc:2229] T b9bea728fc344c749cd9ff72059008d6 P 6bbded853ce94d80b70ff5059443baca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:23:05.547029 32661 raft_consensus.cc:2258] T b9bea728fc344c749cd9ff72059008d6 P 6bbded853ce94d80b70ff5059443baca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:23:05.548341 32664 ts_tablet_manager.cc:1905] T b9bea728fc344c749cd9ff72059008d6 P 0aba19b7cb8a402a822a2fd608e3f697: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:23:05.551352 32664 ts_tablet_manager.cc:1918] T b9bea728fc344c749cd9ff72059008d6 P 0aba19b7cb8a402a822a2fd608e3f697: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 18:23:05.552099 32660 ts_tablet_manager.cc:1905] T b9bea728fc344c749cd9ff72059008d6 P b340acac117a4ec8a029ab8195ea42ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:23:05.554770 32660 ts_tablet_manager.cc:1918] T b9bea728fc344c749cd9ff72059008d6 P b340acac117a4ec8a029ab8195ea42ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 18:23:05.554836 32660 log.cc:1196] T b9bea728fc344c749cd9ff72059008d6 P b340acac117a4ec8a029ab8195ea42ba: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-2/wal/wals/b9bea728fc344c749cd9ff72059008d6
I20260808 18:23:05.555089 32660 ts_tablet_manager.cc:1939] T b9bea728fc344c749cd9ff72059008d6 P b340acac117a4ec8a029ab8195ea42ba: Deleting consensus metadata
I20260808 18:23:05.555989 32661 ts_tablet_manager.cc:1905] T b9bea728fc344c749cd9ff72059008d6 P 6bbded853ce94d80b70ff5059443baca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:23:05.558591 32661 ts_tablet_manager.cc:1918] T b9bea728fc344c749cd9ff72059008d6 P 6bbded853ce94d80b70ff5059443baca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 18:23:05.558657 32661 log.cc:1196] T b9bea728fc344c749cd9ff72059008d6 P 6bbded853ce94d80b70ff5059443baca: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-1/wal/wals/b9bea728fc344c749cd9ff72059008d6
I20260808 18:23:05.558902 32661 ts_tablet_manager.cc:1939] T b9bea728fc344c749cd9ff72059008d6 P 6bbded853ce94d80b70ff5059443baca: Deleting consensus metadata
I20260808 18:23:05.559618 32664 log.cc:1196] T b9bea728fc344c749cd9ff72059008d6 P 0aba19b7cb8a402a822a2fd608e3f697: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-0/wal/wals/b9bea728fc344c749cd9ff72059008d6
I20260808 18:23:05.559937 32664 ts_tablet_manager.cc:1939] T b9bea728fc344c749cd9ff72059008d6 P 0aba19b7cb8a402a822a2fd608e3f697: Deleting consensus metadata
I20260808 18:23:05.560199 32038 catalog_manager.cc:4841] TS 6bbded853ce94d80b70ff5059443baca (127.23.72.130:43777): tablet b9bea728fc344c749cd9ff72059008d6 (table test-table [id=76e73b8109254127bacc3ab8049e0fff]) successfully deleted
I20260808 18:23:05.560542 32038 catalog_manager.cc:4841] TS b340acac117a4ec8a029ab8195ea42ba (127.23.72.131:41059): tablet b9bea728fc344c749cd9ff72059008d6 (table test-table [id=76e73b8109254127bacc3ab8049e0fff]) successfully deleted
I20260808 18:23:05.561147 32037 catalog_manager.cc:4841] TS 0aba19b7cb8a402a822a2fd608e3f697 (127.23.72.129:44001): tablet b9bea728fc344c749cd9ff72059008d6 (table test-table [id=76e73b8109254127bacc3ab8049e0fff]) successfully deleted
Aug 08 18:23:05 dist-test-slave-5k54 krb5kdc[32004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 18:23:05 dist-test-slave-5k54 krb5kdc[32004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:05 dist-test-slave-5k54 krb5kdc[32004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 18:23:05.585662 32683 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:05.577274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36968 (local address 127.23.72.190:40515)
0808 18:23:05.577425 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:05.577430 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:05.577659 (+ 229us) server_negotiation.cc:407] Connection header received
0808 18:23:05.577904 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:05.577908 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:05.577957 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:05.578008 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:05.579103 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:05.579979 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:05.581231 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:05.581601 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:05.581670 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:05.585525 (+ 3855us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36968: BlockingRecv error: recv got EOF from 127.0.0.1:36968 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 08 18:23:05 dist-test-slave-5k54 krb5kdc[32004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM
I20260808 18:23:05.597872 32683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:05.587805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36978 (local address 127.23.72.190:40515)
0808 18:23:05.587919 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:05.587923 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:05.588144 (+ 221us) server_negotiation.cc:407] Connection header received
0808 18:23:05.588251 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:05.588253 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:05.588302 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:05.588406 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:05.589113 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:05.589920 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:05.591054 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:05.591497 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:05.591594 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:05.594426 (+ 2832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:23:05.594430 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:23:05.594433 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:23:05.594459 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:23:05.596164 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:23:05.596615 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:23:05.596619 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:23:05.596621 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:23:05.596708 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:23:05.596979 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:23:05.596982 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:23:05.596984 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:23:05.597200 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:23:05.597339 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:05.597594 (+ 255us) server_negotiation.cc:299] Negotiation successful
0808 18:23:05.597711 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 18:23:05.600618 32051 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36978:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:23:05.601105 32051 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:23:05.606184 32213 tablet_service.cc:1467] Processing CreateTablet for tablet 41509cfbfbe342e590f39980c7f55874 (DEFAULT_TABLE table=test-table [id=18473603205344a5909dc3e27c86e73f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:23:05.606499 32213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41509cfbfbe342e590f39980c7f55874. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:23:05.606526 32524 tablet_service.cc:1467] Processing CreateTablet for tablet 41509cfbfbe342e590f39980c7f55874 (DEFAULT_TABLE table=test-table [id=18473603205344a5909dc3e27c86e73f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:23:05.606525 32362 tablet_service.cc:1467] Processing CreateTablet for tablet 41509cfbfbe342e590f39980c7f55874 (DEFAULT_TABLE table=test-table [id=18473603205344a5909dc3e27c86e73f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:23:05.606808 32362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41509cfbfbe342e590f39980c7f55874. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:23:05.606807 32524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41509cfbfbe342e590f39980c7f55874. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:23:05.608940 32633 tablet_bootstrap.cc:492] T 41509cfbfbe342e590f39980c7f55874 P 0aba19b7cb8a402a822a2fd608e3f697: Bootstrap starting.
I20260808 18:23:05.609246 32635 tablet_bootstrap.cc:492] T 41509cfbfbe342e590f39980c7f55874 P 6bbded853ce94d80b70ff5059443baca: Bootstrap starting.
I20260808 18:23:05.609357 32634 tablet_bootstrap.cc:492] T 41509cfbfbe342e590f39980c7f55874 P b340acac117a4ec8a029ab8195ea42ba: Bootstrap starting.
I20260808 18:23:05.609761 32633 tablet_bootstrap.cc:654] T 41509cfbfbe342e590f39980c7f55874 P 0aba19b7cb8a402a822a2fd608e3f697: Neither blocks nor log segments found. Creating new log.
I20260808 18:23:05.610150 32635 tablet_bootstrap.cc:654] T 41509cfbfbe342e590f39980c7f55874 P 6bbded853ce94d80b70ff5059443baca: Neither blocks nor log segments found. Creating new log.
I20260808 18:23:05.610239 32634 tablet_bootstrap.cc:654] T 41509cfbfbe342e590f39980c7f55874 P b340acac117a4ec8a029ab8195ea42ba: Neither blocks nor log segments found. Creating new log.
I20260808 18:23:05.611083 32633 tablet_bootstrap.cc:492] T 41509cfbfbe342e590f39980c7f55874 P 0aba19b7cb8a402a822a2fd608e3f697: No bootstrap required, opened a new log
I20260808 18:23:05.611172 32633 ts_tablet_manager.cc:1397] T 41509cfbfbe342e590f39980c7f55874 P 0aba19b7cb8a402a822a2fd608e3f697: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 18:23:05.611569 32633 raft_consensus.cc:348] T 41509cfbfbe342e590f39980c7f55874 P 0aba19b7cb8a402a822a2fd608e3f697 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } }
I20260808 18:23:05.611685 32633 raft_consensus.cc:374] T 41509cfbfbe342e590f39980c7f55874 P 0aba19b7cb8a402a822a2fd608e3f697 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:23:05.611716 32633 raft_consensus.cc:729] T 41509cfbfbe342e590f39980c7f55874 P 0aba19b7cb8a402a822a2fd608e3f697 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0aba19b7cb8a402a822a2fd608e3f697, State: Initialized, Role: FOLLOWER
I20260808 18:23:05.611840 32634 tablet_bootstrap.cc:492] T 41509cfbfbe342e590f39980c7f55874 P b340acac117a4ec8a029ab8195ea42ba: No bootstrap required, opened a new log
I20260808 18:23:05.611830 32633 consensus_queue.cc:260] T 41509cfbfbe342e590f39980c7f55874 P 0aba19b7cb8a402a822a2fd608e3f697 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } }
I20260808 18:23:05.611950 32634 ts_tablet_manager.cc:1397] T 41509cfbfbe342e590f39980c7f55874 P b340acac117a4ec8a029ab8195ea42ba: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 18:23:05.612087 32633 ts_tablet_manager.cc:1428] T 41509cfbfbe342e590f39980c7f55874 P 0aba19b7cb8a402a822a2fd608e3f697: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:23:05.612411 32634 raft_consensus.cc:348] T 41509cfbfbe342e590f39980c7f55874 P b340acac117a4ec8a029ab8195ea42ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } }
I20260808 18:23:05.612525 32634 raft_consensus.cc:374] T 41509cfbfbe342e590f39980c7f55874 P b340acac117a4ec8a029ab8195ea42ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:23:05.612562 32634 raft_consensus.cc:729] T 41509cfbfbe342e590f39980c7f55874 P b340acac117a4ec8a029ab8195ea42ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b340acac117a4ec8a029ab8195ea42ba, State: Initialized, Role: FOLLOWER
I20260808 18:23:05.612676 32634 consensus_queue.cc:260] T 41509cfbfbe342e590f39980c7f55874 P b340acac117a4ec8a029ab8195ea42ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } }
I20260808 18:23:05.612951 32634 ts_tablet_manager.cc:1428] T 41509cfbfbe342e590f39980c7f55874 P b340acac117a4ec8a029ab8195ea42ba: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:23:05.612996 32635 tablet_bootstrap.cc:492] T 41509cfbfbe342e590f39980c7f55874 P 6bbded853ce94d80b70ff5059443baca: No bootstrap required, opened a new log
I20260808 18:23:05.613072 32635 ts_tablet_manager.cc:1397] T 41509cfbfbe342e590f39980c7f55874 P 6bbded853ce94d80b70ff5059443baca: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260808 18:23:05.613478 32635 raft_consensus.cc:348] T 41509cfbfbe342e590f39980c7f55874 P 6bbded853ce94d80b70ff5059443baca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } }
I20260808 18:23:05.613577 32635 raft_consensus.cc:374] T 41509cfbfbe342e590f39980c7f55874 P 6bbded853ce94d80b70ff5059443baca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:23:05.613607 32635 raft_consensus.cc:729] T 41509cfbfbe342e590f39980c7f55874 P 6bbded853ce94d80b70ff5059443baca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bbded853ce94d80b70ff5059443baca, State: Initialized, Role: FOLLOWER
I20260808 18:23:05.613714 32635 consensus_queue.cc:260] T 41509cfbfbe342e590f39980c7f55874 P 6bbded853ce94d80b70ff5059443baca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } }
I20260808 18:23:05.613953 32635 ts_tablet_manager.cc:1428] T 41509cfbfbe342e590f39980c7f55874 P 6bbded853ce94d80b70ff5059443baca: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:23:05.808145 32655 raft_consensus.cc:482] T 41509cfbfbe342e590f39980c7f55874 P 0aba19b7cb8a402a822a2fd608e3f697 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:23:05.808328 32655 raft_consensus.cc:504] T 41509cfbfbe342e590f39980c7f55874 P 0aba19b7cb8a402a822a2fd608e3f697 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } }
I20260808 18:23:05.808836 32655 leader_election.cc:288] T 41509cfbfbe342e590f39980c7f55874 P 0aba19b7cb8a402a822a2fd608e3f697 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6bbded853ce94d80b70ff5059443baca (127.23.72.130:43777), b340acac117a4ec8a029ab8195ea42ba (127.23.72.131:41059)
I20260808 18:23:05.809441 32383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41509cfbfbe342e590f39980c7f55874" candidate_uuid: "0aba19b7cb8a402a822a2fd608e3f697" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bbded853ce94d80b70ff5059443baca" is_pre_election: true
I20260808 18:23:05.809458 32546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41509cfbfbe342e590f39980c7f55874" candidate_uuid: "0aba19b7cb8a402a822a2fd608e3f697" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b340acac117a4ec8a029ab8195ea42ba" is_pre_election: true
I20260808 18:23:05.809588 32383 raft_consensus.cc:2454] T 41509cfbfbe342e590f39980c7f55874 P 6bbded853ce94d80b70ff5059443baca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0aba19b7cb8a402a822a2fd608e3f697 in term 0.
I20260808 18:23:05.809645 32546 raft_consensus.cc:2454] T 41509cfbfbe342e590f39980c7f55874 P b340acac117a4ec8a029ab8195ea42ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0aba19b7cb8a402a822a2fd608e3f697 in term 0.
I20260808 18:23:05.810025 32161 leader_election.cc:302] T 41509cfbfbe342e590f39980c7f55874 P 0aba19b7cb8a402a822a2fd608e3f697 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0aba19b7cb8a402a822a2fd608e3f697, b340acac117a4ec8a029ab8195ea42ba; no voters:
I20260808 18:23:05.810225 32655 raft_consensus.cc:2781] T 41509cfbfbe342e590f39980c7f55874 P 0aba19b7cb8a402a822a2fd608e3f697 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:23:05.810308 32655 raft_consensus.cc:482] T 41509cfbfbe342e590f39980c7f55874 P 0aba19b7cb8a402a822a2fd608e3f697 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:23:05.810370 32655 raft_consensus.cc:3037] T 41509cfbfbe342e590f39980c7f55874 P 0aba19b7cb8a402a822a2fd608e3f697 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:23:05.811592 32655 raft_consensus.cc:504] T 41509cfbfbe342e590f39980c7f55874 P 0aba19b7cb8a402a822a2fd608e3f697 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } }
I20260808 18:23:05.811996 32655 leader_election.cc:288] T 41509cfbfbe342e590f39980c7f55874 P 0aba19b7cb8a402a822a2fd608e3f697 [CANDIDATE]: Term 1 election: Requested vote from peers 6bbded853ce94d80b70ff5059443baca (127.23.72.130:43777), b340acac117a4ec8a029ab8195ea42ba (127.23.72.131:41059)
I20260808 18:23:05.812290 32383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41509cfbfbe342e590f39980c7f55874" candidate_uuid: "0aba19b7cb8a402a822a2fd608e3f697" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bbded853ce94d80b70ff5059443baca"
I20260808 18:23:05.812420 32383 raft_consensus.cc:3037] T 41509cfbfbe342e590f39980c7f55874 P 6bbded853ce94d80b70ff5059443baca [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:23:05.812487 32546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41509cfbfbe342e590f39980c7f55874" candidate_uuid: "0aba19b7cb8a402a822a2fd608e3f697" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b340acac117a4ec8a029ab8195ea42ba"
I20260808 18:23:05.812645 32546 raft_consensus.cc:3037] T 41509cfbfbe342e590f39980c7f55874 P b340acac117a4ec8a029ab8195ea42ba [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:23:05.813426 32383 raft_consensus.cc:2454] T 41509cfbfbe342e590f39980c7f55874 P 6bbded853ce94d80b70ff5059443baca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0aba19b7cb8a402a822a2fd608e3f697 in term 1.
I20260808 18:23:05.813717 32161 leader_election.cc:302] T 41509cfbfbe342e590f39980c7f55874 P 0aba19b7cb8a402a822a2fd608e3f697 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0aba19b7cb8a402a822a2fd608e3f697, 6bbded853ce94d80b70ff5059443baca; no voters:
I20260808 18:23:05.813707 32546 raft_consensus.cc:2454] T 41509cfbfbe342e590f39980c7f55874 P b340acac117a4ec8a029ab8195ea42ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0aba19b7cb8a402a822a2fd608e3f697 in term 1.
I20260808 18:23:05.813875 32655 raft_consensus.cc:2781] T 41509cfbfbe342e590f39980c7f55874 P 0aba19b7cb8a402a822a2fd608e3f697 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:23:05.814007 32655 raft_consensus.cc:686] T 41509cfbfbe342e590f39980c7f55874 P 0aba19b7cb8a402a822a2fd608e3f697 [term 1 LEADER]: Becoming Leader. State: Replica: 0aba19b7cb8a402a822a2fd608e3f697, State: Running, Role: LEADER
I20260808 18:23:05.814154 32655 consensus_queue.cc:237] T 41509cfbfbe342e590f39980c7f55874 P 0aba19b7cb8a402a822a2fd608e3f697 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } }
I20260808 18:23:05.815970 32050 catalog_manager.cc:5495] T 41509cfbfbe342e590f39980c7f55874 P 0aba19b7cb8a402a822a2fd608e3f697 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0aba19b7cb8a402a822a2fd608e3f697 (127.23.72.129). New cstate: current_term: 1 leader_uuid: "0aba19b7cb8a402a822a2fd608e3f697" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } health_report { overall_health: UNKNOWN } } }
I20260808 18:23:05.854158 32630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:05.851314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54386 (local address 127.23.72.129:44001)
0808 18:23:05.851484 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:05.851489 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:05.851507 (+ 18us) server_negotiation.cc:407] Connection header received
0808 18:23:05.851560 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:05.851562 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:05.851607 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:05.851674 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:23:05.851942 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:05.852445 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:05.853274 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:05.853491 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:05.853571 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:05.853787 (+ 216us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:23:05.853853 (+ 66us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:23:05.853905 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:05.854008 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 18:23:05.854050 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 18:23:05.856333 32383 raft_consensus.cc:1261] T 41509cfbfbe342e590f39980c7f55874 P 6bbded853ce94d80b70ff5059443baca [term 1 FOLLOWER]: Refusing update from remote peer 0aba19b7cb8a402a822a2fd608e3f697: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:23:05.856387 32546 raft_consensus.cc:1261] T 41509cfbfbe342e590f39980c7f55874 P b340acac117a4ec8a029ab8195ea42ba [term 1 FOLLOWER]: Refusing update from remote peer 0aba19b7cb8a402a822a2fd608e3f697: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:23:05.856786 32646 consensus_queue.cc:1035] T 41509cfbfbe342e590f39980c7f55874 P 0aba19b7cb8a402a822a2fd608e3f697 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:23:05.856942 32655 consensus_queue.cc:1035] T 41509cfbfbe342e590f39980c7f55874 P 0aba19b7cb8a402a822a2fd608e3f697 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:23:05.864954 32052 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36978:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:23:05.865083 32052 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36978:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:23:05.866952 32052 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f0f38b8754bb4ca6a2ec3310e69c9c55: Sending DeleteTablet for 3 replicas of tablet 41509cfbfbe342e590f39980c7f55874
I20260808 18:23:05.867497 32213 tablet_service.cc:1514] Processing DeleteTablet for tablet 41509cfbfbe342e590f39980c7f55874 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:23:05 UTC) from {username='oryx', principal='oryx/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:54354
I20260808 18:23:05.867535 32362 tablet_service.cc:1514] Processing DeleteTablet for tablet 41509cfbfbe342e590f39980c7f55874 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:23:05 UTC) from {username='oryx', principal='oryx/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:57436
I20260808 18:23:05.867624 32524 tablet_service.cc:1514] Processing DeleteTablet for tablet 41509cfbfbe342e590f39980c7f55874 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:23:05 UTC) from {username='oryx', principal='oryx/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:53204
I20260808 18:23:05.867682 32664 tablet_replica.cc:330] T 41509cfbfbe342e590f39980c7f55874 P 0aba19b7cb8a402a822a2fd608e3f697: stopping tablet replica
I20260808 18:23:05.867799 32660 tablet_replica.cc:330] T 41509cfbfbe342e590f39980c7f55874 P b340acac117a4ec8a029ab8195ea42ba: stopping tablet replica
I20260808 18:23:05.867851 32664 raft_consensus.cc:2229] T 41509cfbfbe342e590f39980c7f55874 P 0aba19b7cb8a402a822a2fd608e3f697 [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:23:05.867931 32660 raft_consensus.cc:2229] T 41509cfbfbe342e590f39980c7f55874 P b340acac117a4ec8a029ab8195ea42ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:23:05.868026 32664 raft_consensus.cc:2258] T 41509cfbfbe342e590f39980c7f55874 P 0aba19b7cb8a402a822a2fd608e3f697 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:23:05.868054 32660 raft_consensus.cc:2258] T 41509cfbfbe342e590f39980c7f55874 P b340acac117a4ec8a029ab8195ea42ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:23:05.868213 32661 tablet_replica.cc:330] T 41509cfbfbe342e590f39980c7f55874 P 6bbded853ce94d80b70ff5059443baca: stopping tablet replica
I20260808 18:23:05.868337 32661 raft_consensus.cc:2229] T 41509cfbfbe342e590f39980c7f55874 P 6bbded853ce94d80b70ff5059443baca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:23:05.868468 32661 raft_consensus.cc:2258] T 41509cfbfbe342e590f39980c7f55874 P 6bbded853ce94d80b70ff5059443baca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:23:05.868799 32660 ts_tablet_manager.cc:1905] T 41509cfbfbe342e590f39980c7f55874 P b340acac117a4ec8a029ab8195ea42ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:23:05.868784 32052 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36978:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 18:23:05.869117 32664 ts_tablet_manager.cc:1905] T 41509cfbfbe342e590f39980c7f55874 P 0aba19b7cb8a402a822a2fd608e3f697: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 18:23:05.869179 32052 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:23:05.870265 32661 ts_tablet_manager.cc:1905] T 41509cfbfbe342e590f39980c7f55874 P 6bbded853ce94d80b70ff5059443baca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:23:05.871393 32660 ts_tablet_manager.cc:1918] T 41509cfbfbe342e590f39980c7f55874 P b340acac117a4ec8a029ab8195ea42ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:23:05.871506 32660 log.cc:1196] T 41509cfbfbe342e590f39980c7f55874 P b340acac117a4ec8a029ab8195ea42ba: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-2/wal/wals/41509cfbfbe342e590f39980c7f55874
I20260808 18:23:05.871575 32664 ts_tablet_manager.cc:1918] T 41509cfbfbe342e590f39980c7f55874 P 0aba19b7cb8a402a822a2fd608e3f697: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:23:05.871647 32664 log.cc:1196] T 41509cfbfbe342e590f39980c7f55874 P 0aba19b7cb8a402a822a2fd608e3f697: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-0/wal/wals/41509cfbfbe342e590f39980c7f55874
I20260808 18:23:05.871774 32660 ts_tablet_manager.cc:1939] T 41509cfbfbe342e590f39980c7f55874 P b340acac117a4ec8a029ab8195ea42ba: Deleting consensus metadata
I20260808 18:23:05.871930 32664 ts_tablet_manager.cc:1939] T 41509cfbfbe342e590f39980c7f55874 P 0aba19b7cb8a402a822a2fd608e3f697: Deleting consensus metadata
I20260808 18:23:05.872478 32038 catalog_manager.cc:4841] TS b340acac117a4ec8a029ab8195ea42ba (127.23.72.131:41059): tablet 41509cfbfbe342e590f39980c7f55874 (table test-table [id=18473603205344a5909dc3e27c86e73f]) successfully deleted
I20260808 18:23:05.872704 32661 ts_tablet_manager.cc:1918] T 41509cfbfbe342e590f39980c7f55874 P 6bbded853ce94d80b70ff5059443baca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:23:05.872737 32037 catalog_manager.cc:4841] TS 0aba19b7cb8a402a822a2fd608e3f697 (127.23.72.129:44001): tablet 41509cfbfbe342e590f39980c7f55874 (table test-table [id=18473603205344a5909dc3e27c86e73f]) successfully deleted
I20260808 18:23:05.872809 32661 log.cc:1196] T 41509cfbfbe342e590f39980c7f55874 P 6bbded853ce94d80b70ff5059443baca: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-1/wal/wals/41509cfbfbe342e590f39980c7f55874
I20260808 18:23:05.873050 32661 ts_tablet_manager.cc:1939] T 41509cfbfbe342e590f39980c7f55874 P 6bbded853ce94d80b70ff5059443baca: Deleting consensus metadata
I20260808 18:23:05.874536 32213 tablet_service.cc:1467] Processing CreateTablet for tablet 401b9035905b456fba2914b5dec1b512 (DEFAULT_TABLE table=test-table [id=4a1789d67cc74455b53f0348e8d92c00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:23:05.874788 32213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 401b9035905b456fba2914b5dec1b512. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:23:05.874944 32362 tablet_service.cc:1467] Processing CreateTablet for tablet 401b9035905b456fba2914b5dec1b512 (DEFAULT_TABLE table=test-table [id=4a1789d67cc74455b53f0348e8d92c00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:23:05.874919 32524 tablet_service.cc:1467] Processing CreateTablet for tablet 401b9035905b456fba2914b5dec1b512 (DEFAULT_TABLE table=test-table [id=4a1789d67cc74455b53f0348e8d92c00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:23:05.875170 32362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 401b9035905b456fba2914b5dec1b512. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:23:05.875188 32524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 401b9035905b456fba2914b5dec1b512. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:23:05.876852 32634 tablet_bootstrap.cc:492] T 401b9035905b456fba2914b5dec1b512 P b340acac117a4ec8a029ab8195ea42ba: Bootstrap starting.
I20260808 18:23:05.877061 32633 tablet_bootstrap.cc:492] T 401b9035905b456fba2914b5dec1b512 P 0aba19b7cb8a402a822a2fd608e3f697: Bootstrap starting.
I20260808 18:23:05.877452 32038 catalog_manager.cc:4841] TS 6bbded853ce94d80b70ff5059443baca (127.23.72.130:43777): tablet 41509cfbfbe342e590f39980c7f55874 (table test-table [id=18473603205344a5909dc3e27c86e73f]) successfully deleted
I20260808 18:23:05.877763 32634 tablet_bootstrap.cc:654] T 401b9035905b456fba2914b5dec1b512 P b340acac117a4ec8a029ab8195ea42ba: Neither blocks nor log segments found. Creating new log.
I20260808 18:23:05.877878 32633 tablet_bootstrap.cc:654] T 401b9035905b456fba2914b5dec1b512 P 0aba19b7cb8a402a822a2fd608e3f697: Neither blocks nor log segments found. Creating new log.
I20260808 18:23:05.878374 32635 tablet_bootstrap.cc:492] T 401b9035905b456fba2914b5dec1b512 P 6bbded853ce94d80b70ff5059443baca: Bootstrap starting.
I20260808 18:23:05.878916 32633 tablet_bootstrap.cc:492] T 401b9035905b456fba2914b5dec1b512 P 0aba19b7cb8a402a822a2fd608e3f697: No bootstrap required, opened a new log
I20260808 18:23:05.878906 32634 tablet_bootstrap.cc:492] T 401b9035905b456fba2914b5dec1b512 P b340acac117a4ec8a029ab8195ea42ba: No bootstrap required, opened a new log
I20260808 18:23:05.879002 32634 ts_tablet_manager.cc:1397] T 401b9035905b456fba2914b5dec1b512 P b340acac117a4ec8a029ab8195ea42ba: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 18:23:05.879002 32633 ts_tablet_manager.cc:1397] T 401b9035905b456fba2914b5dec1b512 P 0aba19b7cb8a402a822a2fd608e3f697: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 18:23:05.879417 32635 tablet_bootstrap.cc:654] T 401b9035905b456fba2914b5dec1b512 P 6bbded853ce94d80b70ff5059443baca: Neither blocks nor log segments found. Creating new log.
I20260808 18:23:05.879374 32634 raft_consensus.cc:348] T 401b9035905b456fba2914b5dec1b512 P b340acac117a4ec8a029ab8195ea42ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } }
I20260808 18:23:05.879374 32633 raft_consensus.cc:348] T 401b9035905b456fba2914b5dec1b512 P 0aba19b7cb8a402a822a2fd608e3f697 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } }
I20260808 18:23:05.879508 32633 raft_consensus.cc:374] T 401b9035905b456fba2914b5dec1b512 P 0aba19b7cb8a402a822a2fd608e3f697 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:23:05.879508 32634 raft_consensus.cc:374] T 401b9035905b456fba2914b5dec1b512 P b340acac117a4ec8a029ab8195ea42ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:23:05.879544 32634 raft_consensus.cc:729] T 401b9035905b456fba2914b5dec1b512 P b340acac117a4ec8a029ab8195ea42ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b340acac117a4ec8a029ab8195ea42ba, State: Initialized, Role: FOLLOWER
I20260808 18:23:05.879544 32633 raft_consensus.cc:729] T 401b9035905b456fba2914b5dec1b512 P 0aba19b7cb8a402a822a2fd608e3f697 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0aba19b7cb8a402a822a2fd608e3f697, State: Initialized, Role: FOLLOWER
I20260808 18:23:05.879659 32634 consensus_queue.cc:260] T 401b9035905b456fba2914b5dec1b512 P b340acac117a4ec8a029ab8195ea42ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } }
I20260808 18:23:05.879657 32633 consensus_queue.cc:260] T 401b9035905b456fba2914b5dec1b512 P 0aba19b7cb8a402a822a2fd608e3f697 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } }
I20260808 18:23:05.879917 32634 ts_tablet_manager.cc:1428] T 401b9035905b456fba2914b5dec1b512 P b340acac117a4ec8a029ab8195ea42ba: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:23:05.879935 32633 ts_tablet_manager.cc:1428] T 401b9035905b456fba2914b5dec1b512 P 0aba19b7cb8a402a822a2fd608e3f697: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:23:05.881130 32635 tablet_bootstrap.cc:492] T 401b9035905b456fba2914b5dec1b512 P 6bbded853ce94d80b70ff5059443baca: No bootstrap required, opened a new log
I20260808 18:23:05.881219 32635 ts_tablet_manager.cc:1397] T 401b9035905b456fba2914b5dec1b512 P 6bbded853ce94d80b70ff5059443baca: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260808 18:23:05.881699 32635 raft_consensus.cc:348] T 401b9035905b456fba2914b5dec1b512 P 6bbded853ce94d80b70ff5059443baca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } }
I20260808 18:23:05.881824 32635 raft_consensus.cc:374] T 401b9035905b456fba2914b5dec1b512 P 6bbded853ce94d80b70ff5059443baca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:23:05.881868 32635 raft_consensus.cc:729] T 401b9035905b456fba2914b5dec1b512 P 6bbded853ce94d80b70ff5059443baca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bbded853ce94d80b70ff5059443baca, State: Initialized, Role: FOLLOWER
I20260808 18:23:05.882015 32635 consensus_queue.cc:260] T 401b9035905b456fba2914b5dec1b512 P 6bbded853ce94d80b70ff5059443baca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } }
I20260808 18:23:05.882297 32635 ts_tablet_manager.cc:1428] T 401b9035905b456fba2914b5dec1b512 P 6bbded853ce94d80b70ff5059443baca: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 18:23:06.031219 32646 raft_consensus.cc:482] T 401b9035905b456fba2914b5dec1b512 P 0aba19b7cb8a402a822a2fd608e3f697 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:23:06.031350 32646 raft_consensus.cc:504] T 401b9035905b456fba2914b5dec1b512 P 0aba19b7cb8a402a822a2fd608e3f697 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } }
I20260808 18:23:06.031853 32646 leader_election.cc:288] T 401b9035905b456fba2914b5dec1b512 P 0aba19b7cb8a402a822a2fd608e3f697 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6bbded853ce94d80b70ff5059443baca (127.23.72.130:43777), b340acac117a4ec8a029ab8195ea42ba (127.23.72.131:41059)
I20260808 18:23:06.032356 32546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401b9035905b456fba2914b5dec1b512" candidate_uuid: "0aba19b7cb8a402a822a2fd608e3f697" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b340acac117a4ec8a029ab8195ea42ba" is_pre_election: true
I20260808 18:23:06.032356 32383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401b9035905b456fba2914b5dec1b512" candidate_uuid: "0aba19b7cb8a402a822a2fd608e3f697" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bbded853ce94d80b70ff5059443baca" is_pre_election: true
I20260808 18:23:06.032513 32546 raft_consensus.cc:2454] T 401b9035905b456fba2914b5dec1b512 P b340acac117a4ec8a029ab8195ea42ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0aba19b7cb8a402a822a2fd608e3f697 in term 0.
I20260808 18:23:06.032513 32383 raft_consensus.cc:2454] T 401b9035905b456fba2914b5dec1b512 P 6bbded853ce94d80b70ff5059443baca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0aba19b7cb8a402a822a2fd608e3f697 in term 0.
I20260808 18:23:06.032790 32161 leader_election.cc:302] T 401b9035905b456fba2914b5dec1b512 P 0aba19b7cb8a402a822a2fd608e3f697 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0aba19b7cb8a402a822a2fd608e3f697, 6bbded853ce94d80b70ff5059443baca; no voters:
I20260808 18:23:06.032972 32646 raft_consensus.cc:2781] T 401b9035905b456fba2914b5dec1b512 P 0aba19b7cb8a402a822a2fd608e3f697 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:23:06.033032 32646 raft_consensus.cc:482] T 401b9035905b456fba2914b5dec1b512 P 0aba19b7cb8a402a822a2fd608e3f697 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:23:06.033082 32646 raft_consensus.cc:3037] T 401b9035905b456fba2914b5dec1b512 P 0aba19b7cb8a402a822a2fd608e3f697 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:23:06.034062 32646 raft_consensus.cc:504] T 401b9035905b456fba2914b5dec1b512 P 0aba19b7cb8a402a822a2fd608e3f697 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } }
I20260808 18:23:06.034374 32646 leader_election.cc:288] T 401b9035905b456fba2914b5dec1b512 P 0aba19b7cb8a402a822a2fd608e3f697 [CANDIDATE]: Term 1 election: Requested vote from peers 6bbded853ce94d80b70ff5059443baca (127.23.72.130:43777), b340acac117a4ec8a029ab8195ea42ba (127.23.72.131:41059)
I20260808 18:23:06.034694 32383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401b9035905b456fba2914b5dec1b512" candidate_uuid: "0aba19b7cb8a402a822a2fd608e3f697" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bbded853ce94d80b70ff5059443baca"
I20260808 18:23:06.034811 32383 raft_consensus.cc:3037] T 401b9035905b456fba2914b5dec1b512 P 6bbded853ce94d80b70ff5059443baca [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:23:06.034919 32546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401b9035905b456fba2914b5dec1b512" candidate_uuid: "0aba19b7cb8a402a822a2fd608e3f697" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b340acac117a4ec8a029ab8195ea42ba"
I20260808 18:23:06.035080 32546 raft_consensus.cc:3037] T 401b9035905b456fba2914b5dec1b512 P b340acac117a4ec8a029ab8195ea42ba [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:23:06.035872 32383 raft_consensus.cc:2454] T 401b9035905b456fba2914b5dec1b512 P 6bbded853ce94d80b70ff5059443baca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0aba19b7cb8a402a822a2fd608e3f697 in term 1.
I20260808 18:23:06.036168 32161 leader_election.cc:302] T 401b9035905b456fba2914b5dec1b512 P 0aba19b7cb8a402a822a2fd608e3f697 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0aba19b7cb8a402a822a2fd608e3f697, 6bbded853ce94d80b70ff5059443baca; no voters:
I20260808 18:23:06.036320 32646 raft_consensus.cc:2781] T 401b9035905b456fba2914b5dec1b512 P 0aba19b7cb8a402a822a2fd608e3f697 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:23:06.036360 32546 raft_consensus.cc:2454] T 401b9035905b456fba2914b5dec1b512 P b340acac117a4ec8a029ab8195ea42ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0aba19b7cb8a402a822a2fd608e3f697 in term 1.
I20260808 18:23:06.036433 32646 raft_consensus.cc:686] T 401b9035905b456fba2914b5dec1b512 P 0aba19b7cb8a402a822a2fd608e3f697 [term 1 LEADER]: Becoming Leader. State: Replica: 0aba19b7cb8a402a822a2fd608e3f697, State: Running, Role: LEADER
I20260808 18:23:06.036573 32646 consensus_queue.cc:237] T 401b9035905b456fba2914b5dec1b512 P 0aba19b7cb8a402a822a2fd608e3f697 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } }
I20260808 18:23:06.038219 32050 catalog_manager.cc:5495] T 401b9035905b456fba2914b5dec1b512 P 0aba19b7cb8a402a822a2fd608e3f697 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0aba19b7cb8a402a822a2fd608e3f697 (127.23.72.129). New cstate: current_term: 1 leader_uuid: "0aba19b7cb8a402a822a2fd608e3f697" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aba19b7cb8a402a822a2fd608e3f697" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44001 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 } health_report { overall_health: UNKNOWN } } }
I20260808 18:23:06.085306 32630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:06.082159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:47267 (local address 127.23.72.129:44001)
0808 18:23:06.082270 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:06.082274 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:06.082426 (+ 152us) server_negotiation.cc:407] Connection header received
0808 18:23:06.082515 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:06.082517 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:06.082557 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:06.082622 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:23:06.082900 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:06.083404 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:06.084382 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:06.084582 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:06.084660 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:06.084936 (+ 276us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:23:06.084997 (+ 61us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:23:06.085050 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:06.085135 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 18:23:06.085173 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 18:23:06.085330 32734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:06.082071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:44001 (local address 127.23.72.129:47267)
0808 18:23:06.082348 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:06.082360 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:06.082448 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:06.082676 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:06.082679 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:06.082686 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 18:23:06.082792 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:06.082797 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:06.083564 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:06.083567 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:06.084263 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:06.084272 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:06.084686 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:06.084689 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:06.084753 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:06.084829 (+ 76us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 18:23:06.085073 (+ 244us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 18:23:06.085078 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 18:23:06.085130 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 18:23:06.085197 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":74,"threads_started":1}
I20260808 18:23:06.091594 32383 raft_consensus.cc:1261] T 401b9035905b456fba2914b5dec1b512 P 6bbded853ce94d80b70ff5059443baca [term 1 FOLLOWER]: Refusing update from remote peer 0aba19b7cb8a402a822a2fd608e3f697: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:23:06.091589 32546 raft_consensus.cc:1261] T 401b9035905b456fba2914b5dec1b512 P b340acac117a4ec8a029ab8195ea42ba [term 1 FOLLOWER]: Refusing update from remote peer 0aba19b7cb8a402a822a2fd608e3f697: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:23:06.092020 32646 consensus_queue.cc:1035] T 401b9035905b456fba2914b5dec1b512 P 0aba19b7cb8a402a822a2fd608e3f697 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b340acac117a4ec8a029ab8195ea42ba" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 41059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:23:06.092204 32641 consensus_queue.cc:1035] T 401b9035905b456fba2914b5dec1b512 P 0aba19b7cb8a402a822a2fd608e3f697 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bbded853ce94d80b70ff5059443baca" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 43777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:23:06.125818 32050 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36978:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:23:06.126003 32050 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36978:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:23:06.128019 32050 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f0f38b8754bb4ca6a2ec3310e69c9c55: Sending DeleteTablet for 3 replicas of tablet 401b9035905b456fba2914b5dec1b512
I20260808 18:23:06.128561 32213 tablet_service.cc:1514] Processing DeleteTablet for tablet 401b9035905b456fba2914b5dec1b512 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:23:06 UTC) from {username='oryx', principal='oryx/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:54354
I20260808 18:23:06.128647 32362 tablet_service.cc:1514] Processing DeleteTablet for tablet 401b9035905b456fba2914b5dec1b512 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:23:06 UTC) from {username='oryx', principal='oryx/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:57436
I20260808 18:23:06.128647 32524 tablet_service.cc:1514] Processing DeleteTablet for tablet 401b9035905b456fba2914b5dec1b512 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:23:06 UTC) from {username='oryx', principal='oryx/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:53204
I20260808 18:23:06.128741 32664 tablet_replica.cc:330] T 401b9035905b456fba2914b5dec1b512 P 0aba19b7cb8a402a822a2fd608e3f697: stopping tablet replica
I20260808 18:23:06.128808 32661 tablet_replica.cc:330] T 401b9035905b456fba2914b5dec1b512 P 6bbded853ce94d80b70ff5059443baca: stopping tablet replica
I20260808 18:23:06.128808 32660 tablet_replica.cc:330] T 401b9035905b456fba2914b5dec1b512 P b340acac117a4ec8a029ab8195ea42ba: stopping tablet replica
I20260808 18:23:06.128862 32664 raft_consensus.cc:2229] T 401b9035905b456fba2914b5dec1b512 P 0aba19b7cb8a402a822a2fd608e3f697 [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:23:06.128906 32661 raft_consensus.cc:2229] T 401b9035905b456fba2914b5dec1b512 P 6bbded853ce94d80b70ff5059443baca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:23:06.128906 32660 raft_consensus.cc:2229] T 401b9035905b456fba2914b5dec1b512 P b340acac117a4ec8a029ab8195ea42ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:23:06.128999 32664 raft_consensus.cc:2258] T 401b9035905b456fba2914b5dec1b512 P 0aba19b7cb8a402a822a2fd608e3f697 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:23:06.129004 32660 raft_consensus.cc:2258] T 401b9035905b456fba2914b5dec1b512 P b340acac117a4ec8a029ab8195ea42ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:23:06.129004 32661 raft_consensus.cc:2258] T 401b9035905b456fba2914b5dec1b512 P 6bbded853ce94d80b70ff5059443baca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:23:06.129849 32661 ts_tablet_manager.cc:1905] T 401b9035905b456fba2914b5dec1b512 P 6bbded853ce94d80b70ff5059443baca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:23:06.129848 32660 ts_tablet_manager.cc:1905] T 401b9035905b456fba2914b5dec1b512 P b340acac117a4ec8a029ab8195ea42ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:23:06.130012 32664 ts_tablet_manager.cc:1905] T 401b9035905b456fba2914b5dec1b512 P 0aba19b7cb8a402a822a2fd608e3f697: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:23:06.132313 32664 ts_tablet_manager.cc:1918] T 401b9035905b456fba2914b5dec1b512 P 0aba19b7cb8a402a822a2fd608e3f697: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 18:23:06.132314 32661 ts_tablet_manager.cc:1918] T 401b9035905b456fba2914b5dec1b512 P 6bbded853ce94d80b70ff5059443baca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 18:23:06.132314 32660 ts_tablet_manager.cc:1918] T 401b9035905b456fba2914b5dec1b512 P b340acac117a4ec8a029ab8195ea42ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 18:23:06.132411 32660 log.cc:1196] T 401b9035905b456fba2914b5dec1b512 P b340acac117a4ec8a029ab8195ea42ba: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-2/wal/wals/401b9035905b456fba2914b5dec1b512
I20260808 18:23:06.132411 32661 log.cc:1196] T 401b9035905b456fba2914b5dec1b512 P 6bbded853ce94d80b70ff5059443baca: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-1/wal/wals/401b9035905b456fba2914b5dec1b512
I20260808 18:23:06.132416 32664 log.cc:1196] T 401b9035905b456fba2914b5dec1b512 P 0aba19b7cb8a402a822a2fd608e3f697: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213345431840-23842-0/minicluster-data/ts-0/wal/wals/401b9035905b456fba2914b5dec1b512
I20260808 18:23:06.132695 32661 ts_tablet_manager.cc:1939] T 401b9035905b456fba2914b5dec1b512 P 6bbded853ce94d80b70ff5059443baca: Deleting consensus metadata
I20260808 18:23:06.132695 32660 ts_tablet_manager.cc:1939] T 401b9035905b456fba2914b5dec1b512 P b340acac117a4ec8a029ab8195ea42ba: Deleting consensus metadata
I20260808 18:23:06.132745 32664 ts_tablet_manager.cc:1939] T 401b9035905b456fba2914b5dec1b512 P 0aba19b7cb8a402a822a2fd608e3f697: Deleting consensus metadata
I20260808 18:23:06.133466 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 32139
I20260808 18:23:06.133459 32037 catalog_manager.cc:4841] TS 0aba19b7cb8a402a822a2fd608e3f697 (127.23.72.129:44001): tablet 401b9035905b456fba2914b5dec1b512 (table test-table [id=4a1789d67cc74455b53f0348e8d92c00]) successfully deleted
I20260808 18:23:06.133494 32038 catalog_manager.cc:4841] TS 6bbded853ce94d80b70ff5059443baca (127.23.72.130:43777): tablet 401b9035905b456fba2914b5dec1b512 (table test-table [id=4a1789d67cc74455b53f0348e8d92c00]) successfully deleted
I20260808 18:23:06.133652 32038 catalog_manager.cc:4841] TS b340acac117a4ec8a029ab8195ea42ba (127.23.72.131:41059): tablet 401b9035905b456fba2914b5dec1b512 (table test-table [id=4a1789d67cc74455b53f0348e8d92c00]) successfully deleted
I20260808 18:23:06.143734 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 32295
I20260808 18:23:06.150691 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 32447
I20260808 18:23:06.158938 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 32017
2026-08-08T18:23:06Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3632 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213345431840-23842-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:23:06 dist-test-slave-5k54 krb5kdc[32740](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:23:06 dist-test-slave-5k54 krb5kdc[32740](info): set up 2 sockets
Aug 08 18:23:06 dist-test-slave-5k54 krb5kdc[32740](info): commencing operation
krb5kdc: starting...
W20260808 18:23:07.256793 23842 mini_kdc.cc:121] Time spent starting KDC: real 1.073s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Aug 08 18:23:07 dist-test-slave-5k54 krb5kdc[32740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:23:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:23:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.72.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
I20260808 18:23:07.382874 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:34943
--webserver_interface=127.23.72.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:38425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:34943
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:23:07.492259 32755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:23:07.492550 32755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:23:07.492623 32755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:23:07.496526 32755 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:23:07.496603 32755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:23:07.496644 32755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:23:07.496704 32755 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:23:07.496743 32755 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:23:07.501801 32755 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:38425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:34943
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:34943
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.72.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32755
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:23:07.502806 32755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:23:07.503831 32755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 18:23:07.512764 32755 server_base.cc:1034] running on GCE node
W20260808 18:23:07.512729 32762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:23:07.512748 32764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:23:07.512729 32761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:23:07.513584 32755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:23:07.514668 32755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:23:07.515873 32755 hybrid_clock.cc:648] HybridClock initialized: now 1786213387515843 us; error 39 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-08T18:23:07Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1368 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:23:07 dist-test-slave-5k54 krb5kdc[301](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:23:07 dist-test-slave-5k54 krb5kdc[301](info): set up 2 sockets
Aug 08 18:23:07 dist-test-slave-5k54 krb5kdc[301](info): commencing operation
krb5kdc: starting...
W20260808 18:23:08.608778 23842 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Aug 08 18:23:08 dist-test-slave-5k54 krb5kdc[301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:23:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:23:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.72.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
I20260808 18:23:08.738421 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:41351
--webserver_interface=127.23.72.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:35157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:41351
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:23:08.849263 315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:23:08.849557 315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:23:08.849627 315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:23:08.853721 315 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:23:08.853801 315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:23:08.853838 315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:23:08.853904 315 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:23:08.853945 315 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:23:08.859112 315 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:35157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:41351
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:41351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.72.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:23:08.860201 315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:23:08.861250 315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:23:08.869638 321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:23:08.869671 320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:23:08.869910 323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:23:08.869792 315 server_base.cc:1034] running on GCE node
I20260808 18:23:08.870713 315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:23:08.871881 315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:23:08.873045 315 hybrid_clock.cc:648] HybridClock initialized: now 1786213388873019 us; error 43 us; skew 500 ppm
Aug 08 18:23:08 dist-test-slave-5k54 krb5kdc[301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213388, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:23:08.876127 315 init.cc:377] Logged in from keytab as kudu/127.23.72.190@KRBTEST.COM (short username kudu)
I20260808 18:23:08.877213 315 webserver.cc:460] Webserver started at http://127.23.72.190:41085/ using document root <none> and password file <none>
I20260808 18:23:08.877924 315 fs_manager.cc:359] Metadata directory not provided
I20260808 18:23:08.877991 315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:23:08.878237 315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:23:08.880861 315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "a986ee0f4ea045c48dc6682a417e9a51"
format_stamp: "Formatted at 2026-08-08 18:23:08 on dist-test-slave-5k54"
server_key: "d1130aaaeb1238012be2c0bef85d49b0"
server_key_iv: "a6e7488fc65593049824dab92e274981"
server_key_version: "encryptionkey@0"
I20260808 18:23:08.881450 315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "a986ee0f4ea045c48dc6682a417e9a51"
format_stamp: "Formatted at 2026-08-08 18:23:08 on dist-test-slave-5k54"
server_key: "d1130aaaeb1238012be2c0bef85d49b0"
server_key_iv: "a6e7488fc65593049824dab92e274981"
server_key_version: "encryptionkey@0"
I20260808 18:23:08.885197 315 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 18:23:08.887570 330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:08.888751 315 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:23:08.888895 315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "a986ee0f4ea045c48dc6682a417e9a51"
format_stamp: "Formatted at 2026-08-08 18:23:08 on dist-test-slave-5k54"
server_key: "d1130aaaeb1238012be2c0bef85d49b0"
server_key_iv: "a6e7488fc65593049824dab92e274981"
server_key_version: "encryptionkey@0"
I20260808 18:23:08.889003 315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:23:08.902567 315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:23:08.905409 315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:23:08.905604 315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:23:08.913687 315 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.190:41351
I20260808 18:23:08.913717 389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.190:41351 every 8 connection(s)
I20260808 18:23:08.914577 315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:23:08.917289 390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:23:08.922686 390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a986ee0f4ea045c48dc6682a417e9a51: Bootstrap starting.
I20260808 18:23:08.924831 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 315
I20260808 18:23:08.924997 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:23:08.925232 390 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a986ee0f4ea045c48dc6682a417e9a51: Neither blocks nor log segments found. Creating new log.
I20260808 18:23:08.925297 23842 external_mini_cluster.cc:1428] Setting key fb392080c138122b01c8ea94d277639a
I20260808 18:23:08.926105 390 log.cc:824] T 00000000000000000000000000000000 P a986ee0f4ea045c48dc6682a417e9a51: Log is configured to *not* fsync() on all Append() calls
I20260808 18:23:08.928114 390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a986ee0f4ea045c48dc6682a417e9a51: No bootstrap required, opened a new log
I20260808 18:23:08.932389 390 raft_consensus.cc:348] T 00000000000000000000000000000000 P a986ee0f4ea045c48dc6682a417e9a51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a986ee0f4ea045c48dc6682a417e9a51" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 41351 } }
I20260808 18:23:08.932668 390 raft_consensus.cc:374] T 00000000000000000000000000000000 P a986ee0f4ea045c48dc6682a417e9a51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:23:08.932751 390 raft_consensus.cc:729] T 00000000000000000000000000000000 P a986ee0f4ea045c48dc6682a417e9a51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a986ee0f4ea045c48dc6682a417e9a51, State: Initialized, Role: FOLLOWER
I20260808 18:23:08.933280 390 consensus_queue.cc:260] T 00000000000000000000000000000000 P a986ee0f4ea045c48dc6682a417e9a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a986ee0f4ea045c48dc6682a417e9a51" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 41351 } }
I20260808 18:23:08.933462 390 raft_consensus.cc:388] T 00000000000000000000000000000000 P a986ee0f4ea045c48dc6682a417e9a51 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:23:08.933544 390 raft_consensus.cc:482] T 00000000000000000000000000000000 P a986ee0f4ea045c48dc6682a417e9a51 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:23:08.933663 390 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a986ee0f4ea045c48dc6682a417e9a51 [term 0 FOLLOWER]: Advancing to term 1
Aug 08 18:23:08 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:23:08.934687 390 raft_consensus.cc:504] T 00000000000000000000000000000000 P a986ee0f4ea045c48dc6682a417e9a51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a986ee0f4ea045c48dc6682a417e9a51" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 41351 } }
I20260808 18:23:08.935077 390 leader_election.cc:302] T 00000000000000000000000000000000 P a986ee0f4ea045c48dc6682a417e9a51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a986ee0f4ea045c48dc6682a417e9a51; no voters:
I20260808 18:23:08.935478 390 leader_election.cc:288] T 00000000000000000000000000000000 P a986ee0f4ea045c48dc6682a417e9a51 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:23:08.935595 395 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a986ee0f4ea045c48dc6682a417e9a51 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:23:08.935850 395 raft_consensus.cc:686] T 00000000000000000000000000000000 P a986ee0f4ea045c48dc6682a417e9a51 [term 1 LEADER]: Becoming Leader. State: Replica: a986ee0f4ea045c48dc6682a417e9a51, State: Running, Role: LEADER
I20260808 18:23:08.936156 395 consensus_queue.cc:237] T 00000000000000000000000000000000 P a986ee0f4ea045c48dc6682a417e9a51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a986ee0f4ea045c48dc6682a417e9a51" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 41351 } }
I20260808 18:23:08.936944 390 sys_catalog.cc:564] T 00000000000000000000000000000000 P a986ee0f4ea045c48dc6682a417e9a51 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:23:08.937817 397 sys_catalog.cc:455] T 00000000000000000000000000000000 P a986ee0f4ea045c48dc6682a417e9a51 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a986ee0f4ea045c48dc6682a417e9a51. Latest consensus state: current_term: 1 leader_uuid: "a986ee0f4ea045c48dc6682a417e9a51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a986ee0f4ea045c48dc6682a417e9a51" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 41351 } } }
I20260808 18:23:08.937966 397 sys_catalog.cc:458] T 00000000000000000000000000000000 P a986ee0f4ea045c48dc6682a417e9a51 [sys.catalog]: This master's current role is: LEADER
I20260808 18:23:08.938028 396 sys_catalog.cc:455] T 00000000000000000000000000000000 P a986ee0f4ea045c48dc6682a417e9a51 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a986ee0f4ea045c48dc6682a417e9a51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a986ee0f4ea045c48dc6682a417e9a51" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 41351 } } }
I20260808 18:23:08.938113 396 sys_catalog.cc:458] T 00000000000000000000000000000000 P a986ee0f4ea045c48dc6682a417e9a51 [sys.catalog]: This master's current role is: LEADER
I20260808 18:23:08.938395 404 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:23:08.941541 394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:08.926398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43052 (local address 127.23.72.190:41351)
0808 18:23:08.926879 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:08.926904 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:08.926947 (+ 43us) server_negotiation.cc:407] Connection header received
0808 18:23:08.927622 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:08.927640 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:08.927946 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:08.928239 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:08.929094 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:08.930065 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:08.931378 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:08.931940 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:08.932069 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:08.935064 (+ 2995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:23:08.935070 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:23:08.935088 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:23:08.935125 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:23:08.937370 (+ 2245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:23:08.937944 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:23:08.937952 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:23:08.937958 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:23:08.938398 (+ 440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:23:08.938752 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:23:08.938757 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:23:08.938759 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:23:08.939273 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:23:08.939517 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:08.940266 (+ 749us) server_negotiation.cc:299] Negotiation successful
0808 18:23:08.940590 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":164,"threads_started":1}
I20260808 18:23:08.941745 404 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:23:08.947762 404 catalog_manager.cc:1347] Generated new cluster ID: a32ad68aaf8c46a9829951fc0c45be96
I20260808 18:23:08.947832 404 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:23:08.962904 404 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:23:08.963793 404 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:23:08.975275 404 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a986ee0f4ea045c48dc6682a417e9a51: Generated new TSK 0
I20260808 18:23:08.975962 404 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.72.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab.
Entry for principal kudu/127.23.72.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab.
I20260808 18:23:09.008800 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.129:0
--local_ip_for_outbound_sockets=127.23.72.129
--webserver_interface=127.23.72.129
--webserver_port=0
--tserver_master_addrs=127.23.72.190:41351
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab
--principal=kudu/127.23.72.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:35157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:23:09.133385 416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:23:09.133639 416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:23:09.133682 416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:23:09.137740 416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:23:09.137797 416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:23:09.137853 416 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.129
I20260808 18:23:09.143198 416 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:35157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab
--principal=kudu/127.23.72.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.72.129
--webserver_port=0
--tserver_master_addrs=127.23.72.190:41351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.416
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.129
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:23:09.144258 416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:23:09.145296 416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:23:09.155167 422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:23:09.155201 421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:23:09.155202 424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:23:09.155404 416 server_base.cc:1034] running on GCE node
I20260808 18:23:09.155825 416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:23:09.156490 416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:23:09.157655 416 hybrid_clock.cc:648] HybridClock initialized: now 1786213389157624 us; error 41 us; skew 500 ppm
Aug 08 18:23:09 dist-test-slave-5k54 krb5kdc[301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213389, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:23:09.160437 416 init.cc:377] Logged in from keytab as kudu/127.23.72.129@KRBTEST.COM (short username kudu)
I20260808 18:23:09.161320 416 webserver.cc:460] Webserver started at http://127.23.72.129:42349/ using document root <none> and password file <none>
I20260808 18:23:09.161867 416 fs_manager.cc:359] Metadata directory not provided
I20260808 18:23:09.161984 416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:23:09.162217 416 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:23:09.164328 416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-0/data/instance:
uuid: "7861831876c84f38af0aaf70843c598a"
format_stamp: "Formatted at 2026-08-08 18:23:09 on dist-test-slave-5k54"
server_key: "ecd211fc4224cd1b0abc370893641188"
server_key_iv: "4516236220cdcf93cbd9bd2f4674b794"
server_key_version: "encryptionkey@0"
I20260808 18:23:09.164922 416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance:
uuid: "7861831876c84f38af0aaf70843c598a"
format_stamp: "Formatted at 2026-08-08 18:23:09 on dist-test-slave-5k54"
server_key: "ecd211fc4224cd1b0abc370893641188"
server_key_iv: "4516236220cdcf93cbd9bd2f4674b794"
server_key_version: "encryptionkey@0"
I20260808 18:23:09.168615 416 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 18:23:09.170966 434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:09.172180 416 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 18:23:09.172317 416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-0/wal
uuid: "7861831876c84f38af0aaf70843c598a"
format_stamp: "Formatted at 2026-08-08 18:23:09 on dist-test-slave-5k54"
server_key: "ecd211fc4224cd1b0abc370893641188"
server_key_iv: "4516236220cdcf93cbd9bd2f4674b794"
server_key_version: "encryptionkey@0"
I20260808 18:23:09.172429 416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:23:09.190537 416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:23:09.193813 416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:23:09.194029 416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:23:09.194661 416 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:23:09.195583 416 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:23:09.195662 416 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:09.195731 416 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:23:09.195789 416 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:09.205302 416 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.129:43481
I20260808 18:23:09.205341 556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.129:43481 every 8 connection(s)
I20260808 18:23:09.206204 416 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
I20260808 18:23:09.207127 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 416
I20260808 18:23:09.207283 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance
I20260808 18:23:09.207659 23842 external_mini_cluster.cc:1428] Setting key c6f83bd6680ee73120961d22b94e3ba2
Aug 08 18:23:09 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213389, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.72.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.130@KRBTEST.COM" created.
I20260808 18:23:09.221594 394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:09.208098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:43753 (local address 127.23.72.190:41351)
0808 18:23:09.208238 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:09.208242 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:09.209425 (+ 1183us) server_negotiation.cc:407] Connection header received
0808 18:23:09.211548 (+ 2123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:09.211553 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:09.211618 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:09.211723 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:09.212914 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:09.213452 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:09.214691 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:09.214928 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:09.215043 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:09.217803 (+ 2760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:23:09.217807 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:23:09.217810 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:23:09.217843 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:23:09.219401 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:23:09.219979 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:23:09.219983 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:23:09.219985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:23:09.220070 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:23:09.220417 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:23:09.220420 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:23:09.220422 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:23:09.220608 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:23:09.220717 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:09.221270 (+ 553us) server_negotiation.cc:299] Negotiation successful
0808 18:23:09.221409 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 18:23:09.222363 559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:09.208539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.129:43753)
0808 18:23:09.209262 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:09.209334 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:09.211267 (+ 1933us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:09.211922 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:09.211929 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:09.212286 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:09.212755 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:09.212768 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:09.213579 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:09.213582 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:09.214567 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:09.214576 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:09.215080 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:09.215085 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:09.215259 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:09.215962 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:09.215979 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:09.217633 (+ 1654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:23:09.219572 (+ 1939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:23:09.219578 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:23:09.219592 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:23:09.219847 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:23:09.220164 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:23:09.220168 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:23:09.220170 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:23:09.220312 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:23:09.220715 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:23:09.220722 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:23:09.221009 (+ 287us) client_negotiation.cc:769] Sending connection context
0808 18:23:09.221221 (+ 212us) client_negotiation.cc:240] Negotiation successful
0808 18:23:09.221543 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":140,"threads_started":1}
I20260808 18:23:09.223829 557 heartbeater.cc:347] Connected to a master server at 127.23.72.190:41351
I20260808 18:23:09.224090 557 heartbeater.cc:464] Registering TS with master...
I20260808 18:23:09.224567 557 heartbeater.cc:511] Master 127.23.72.190:41351 requested a full tablet report, sending...
I20260808 18:23:09.226091 348 ts_manager.cc:194] Registered new tserver with Master: 7861831876c84f38af0aaf70843c598a (127.23.72.129:43481)
I20260808 18:23:09.227219 348 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.129@KRBTEST.COM'} at 127.23.72.129:43753
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab.
Entry for principal kudu/127.23.72.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab.
I20260808 18:23:09.239195 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.130:0
--local_ip_for_outbound_sockets=127.23.72.130
--webserver_interface=127.23.72.130
--webserver_port=0
--tserver_master_addrs=127.23.72.190:41351
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab
--principal=kudu/127.23.72.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:35157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:23:09.350555 562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:23:09.350808 562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:23:09.350849 562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:23:09.354990 562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:23:09.355055 562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:23:09.355113 562 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.130
I20260808 18:23:09.360258 562 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:35157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab
--principal=kudu/127.23.72.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.72.130
--webserver_port=0
--tserver_master_addrs=127.23.72.190:41351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.562
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.130
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:23:09.361208 562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:23:09.362314 562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 18:23:09.371749 562 server_base.cc:1034] running on GCE node
W20260808 18:23:09.371726 571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:23:09.371737 567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:23:09.371917 568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:23:09.372325 562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:23:09.372953 562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:23:09.374130 562 hybrid_clock.cc:648] HybridClock initialized: now 1786213389374108 us; error 42 us; skew 500 ppm
Aug 08 18:23:09 dist-test-slave-5k54 krb5kdc[301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213389, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:23:09.376984 562 init.cc:377] Logged in from keytab as kudu/127.23.72.130@KRBTEST.COM (short username kudu)
I20260808 18:23:09.377825 562 webserver.cc:460] Webserver started at http://127.23.72.130:39473/ using document root <none> and password file <none>
I20260808 18:23:09.378391 562 fs_manager.cc:359] Metadata directory not provided
I20260808 18:23:09.378485 562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:23:09.378711 562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:23:09.380805 562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-1/data/instance:
uuid: "6b90e1b6cb494402b025b337650c8ca7"
format_stamp: "Formatted at 2026-08-08 18:23:09 on dist-test-slave-5k54"
server_key: "76e0eb51edf8522e3f9f283aa6df1f8b"
server_key_iv: "e9c481ba657a5ff9e44c76e1a6b53901"
server_key_version: "encryptionkey@0"
I20260808 18:23:09.381369 562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance:
uuid: "6b90e1b6cb494402b025b337650c8ca7"
format_stamp: "Formatted at 2026-08-08 18:23:09 on dist-test-slave-5k54"
server_key: "76e0eb51edf8522e3f9f283aa6df1f8b"
server_key_iv: "e9c481ba657a5ff9e44c76e1a6b53901"
server_key_version: "encryptionkey@0"
I20260808 18:23:09.385006 562 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 18:23:09.387346 579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:09.388559 562 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:23:09.388697 562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-1/wal
uuid: "6b90e1b6cb494402b025b337650c8ca7"
format_stamp: "Formatted at 2026-08-08 18:23:09 on dist-test-slave-5k54"
server_key: "76e0eb51edf8522e3f9f283aa6df1f8b"
server_key_iv: "e9c481ba657a5ff9e44c76e1a6b53901"
server_key_version: "encryptionkey@0"
I20260808 18:23:09.388811 562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:23:09.401957 562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:23:09.404887 562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:23:09.405093 562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:23:09.405712 562 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:23:09.406528 562 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:23:09.406600 562 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:09.406675 562 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:23:09.406728 562 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:09.415416 562 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.130:32811
I20260808 18:23:09.415541 700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.130:32811 every 8 connection(s)
I20260808 18:23:09.416417 562 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
Aug 08 18:23:09 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213389, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.130@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:23:09.426193 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 562
I20260808 18:23:09.426348 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance
I20260808 18:23:09.426738 23842 external_mini_cluster.cc:1428] Setting key 5ccac17bc7d2780415b502108cf535a1
I20260808 18:23:09.430675 394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:09.418223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:43295 (local address 127.23.72.190:41351)
0808 18:23:09.418372 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:09.418376 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:09.419197 (+ 821us) server_negotiation.cc:407] Connection header received
0808 18:23:09.420590 (+ 1393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:09.420594 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:09.420655 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:09.420757 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:09.421925 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:09.422463 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:09.423628 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:09.423863 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:09.423973 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:09.426568 (+ 2595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:23:09.426573 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:23:09.426576 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:23:09.426608 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:23:09.428337 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:23:09.428952 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:23:09.428954 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:23:09.428955 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:23:09.429038 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:23:09.429439 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:23:09.429441 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:23:09.429442 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:23:09.429625 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:23:09.429699 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:09.430339 (+ 640us) server_negotiation.cc:299] Negotiation successful
0808 18:23:09.430444 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 18:23:09.431607 703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:09.418606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.130:43295)
0808 18:23:09.419045 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:09.419087 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:09.420345 (+ 1258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:09.420928 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:09.420934 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:09.421292 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:09.421768 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:09.421781 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:09.422596 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:09.422600 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:09.423494 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:09.423503 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:09.424029 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:09.424034 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:09.424207 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:09.424832 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:09.424848 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:09.426353 (+ 1505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:23:09.428446 (+ 2093us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:23:09.428453 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:23:09.428467 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:23:09.428833 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:23:09.429132 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:23:09.429138 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:23:09.429142 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:23:09.429343 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:23:09.429728 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:23:09.429738 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:23:09.430080 (+ 342us) client_negotiation.cc:769] Sending connection context
0808 18:23:09.430301 (+ 221us) client_negotiation.cc:240] Negotiation successful
0808 18:23:09.430682 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":107,"threads_started":1}
I20260808 18:23:09.432833 701 heartbeater.cc:347] Connected to a master server at 127.23.72.190:41351
I20260808 18:23:09.433099 701 heartbeater.cc:464] Registering TS with master...
I20260808 18:23:09.433629 701 heartbeater.cc:511] Master 127.23.72.190:41351 requested a full tablet report, sending...
I20260808 18:23:09.434695 348 ts_manager.cc:194] Registered new tserver with Master: 6b90e1b6cb494402b025b337650c8ca7 (127.23.72.130:32811)
I20260808 18:23:09.435281 348 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.130@KRBTEST.COM'} at 127.23.72.130:43295
WARNING: no policy specified for kudu/127.23.72.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab.
Entry for principal kudu/127.23.72.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab.
I20260808 18:23:09.459613 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.131:0
--local_ip_for_outbound_sockets=127.23.72.131
--webserver_interface=127.23.72.131
--webserver_port=0
--tserver_master_addrs=127.23.72.190:41351
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab
--principal=kudu/127.23.72.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:35157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:23:09.582906 706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:23:09.583169 706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:23:09.583215 706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:23:09.587958 706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:23:09.588019 706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:23:09.588083 706 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.131
I20260808 18:23:09.594048 706 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:35157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab
--principal=kudu/127.23.72.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.72.131
--webserver_port=0
--tserver_master_addrs=127.23.72.190:41351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.706
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.131
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:23:09.595162 706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:23:09.596366 706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:23:09.606240 712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:23:09.606237 711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:23:09.606622 706 server_base.cc:1034] running on GCE node
W20260808 18:23:09.606273 714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:23:09.607086 706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:23:09.607806 706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:23:09.609040 706 hybrid_clock.cc:648] HybridClock initialized: now 1786213389609027 us; error 29 us; skew 500 ppm
Aug 08 18:23:09 dist-test-slave-5k54 krb5kdc[301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213389, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:23:09.612170 706 init.cc:377] Logged in from keytab as kudu/127.23.72.131@KRBTEST.COM (short username kudu)
I20260808 18:23:09.613096 706 webserver.cc:460] Webserver started at http://127.23.72.131:34445/ using document root <none> and password file <none>
I20260808 18:23:09.613695 706 fs_manager.cc:359] Metadata directory not provided
I20260808 18:23:09.613813 706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:23:09.614074 706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:23:09.616401 706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-2/data/instance:
uuid: "46f0bc7d12164a70aa36f32f68fba096"
format_stamp: "Formatted at 2026-08-08 18:23:09 on dist-test-slave-5k54"
server_key: "7cf97126c42ecf276eb12ca79c897745"
server_key_iv: "ca6bc0c3e6c1f0ac4fa4933a7aea02c9"
server_key_version: "encryptionkey@0"
I20260808 18:23:09.617067 706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance:
uuid: "46f0bc7d12164a70aa36f32f68fba096"
format_stamp: "Formatted at 2026-08-08 18:23:09 on dist-test-slave-5k54"
server_key: "7cf97126c42ecf276eb12ca79c897745"
server_key_iv: "ca6bc0c3e6c1f0ac4fa4933a7aea02c9"
server_key_version: "encryptionkey@0"
I20260808 18:23:09.621146 706 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 18:23:09.623848 722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:09.625103 706 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:23:09.625250 706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-2/wal
uuid: "46f0bc7d12164a70aa36f32f68fba096"
format_stamp: "Formatted at 2026-08-08 18:23:09 on dist-test-slave-5k54"
server_key: "7cf97126c42ecf276eb12ca79c897745"
server_key_iv: "ca6bc0c3e6c1f0ac4fa4933a7aea02c9"
server_key_version: "encryptionkey@0"
I20260808 18:23:09.625376 706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:23:09.640607 706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:23:09.643739 706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:23:09.643968 706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:23:09.644591 706 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:23:09.645488 706 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:23:09.645569 706 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:09.645649 706 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:23:09.645696 706 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:09.655391 706 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.131:36489
I20260808 18:23:09.655481 844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.131:36489 every 8 connection(s)
I20260808 18:23:09.656503 706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
I20260808 18:23:09.657346 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 706
I20260808 18:23:09.657518 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance
I20260808 18:23:09.657873 23842 external_mini_cluster.cc:1428] Setting key 56d35b0cee04e50d449b068db6a35d6f
Aug 08 18:23:09 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213389, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.131@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:23:09.670502 394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:09.658728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:56905 (local address 127.23.72.190:41351)
0808 18:23:09.658872 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:09.658875 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:09.659929 (+ 1054us) server_negotiation.cc:407] Connection header received
0808 18:23:09.661394 (+ 1465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:09.661397 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:09.661452 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:09.661527 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:09.662664 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:09.663202 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:09.664360 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:09.664668 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:09.664795 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:09.667171 (+ 2376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:23:09.667174 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:23:09.667177 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:23:09.667204 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:23:09.668446 (+ 1242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:23:09.668951 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:23:09.668954 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:23:09.668956 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:23:09.669050 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:23:09.669392 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:23:09.669397 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:23:09.669398 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:23:09.669584 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:23:09.669697 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:09.670178 (+ 481us) server_negotiation.cc:299] Negotiation successful
0808 18:23:09.670303 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 18:23:09.671499 848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:09.659263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.131:56905)
0808 18:23:09.659783 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:09.659826 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:09.661189 (+ 1363us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:09.661695 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:09.661701 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:09.662052 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:09.662519 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:09.662532 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:09.663328 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:09.663332 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:09.664229 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:09.664239 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:09.664829 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:09.664833 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:09.665009 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:09.665546 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:09.665561 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:09.666997 (+ 1436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:23:09.668564 (+ 1567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:23:09.668569 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:23:09.668583 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:23:09.668840 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:23:09.669143 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:23:09.669147 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:23:09.669149 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:23:09.669288 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:23:09.669694 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:23:09.669701 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:23:09.669942 (+ 241us) client_negotiation.cc:769] Sending connection context
0808 18:23:09.670133 (+ 191us) client_negotiation.cc:240] Negotiation successful
0808 18:23:09.670456 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":178,"threads_started":1}
I20260808 18:23:09.672559 846 heartbeater.cc:347] Connected to a master server at 127.23.72.190:41351
I20260808 18:23:09.672798 846 heartbeater.cc:464] Registering TS with master...
I20260808 18:23:09.673285 846 heartbeater.cc:511] Master 127.23.72.190:41351 requested a full tablet report, sending...
I20260808 18:23:09.674280 348 ts_manager.cc:194] Registered new tserver with Master: 46f0bc7d12164a70aa36f32f68fba096 (127.23.72.131:36489)
I20260808 18:23:09.674839 348 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.131@KRBTEST.COM'} at 127.23.72.131:56905
I20260808 18:23:09.683969 23842 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.23.72.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.72.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.72.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.129.keytab.
Entry for principal oryx/127.23.72.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.129.keytab.
WARNING: no policy specified for oryx/127.23.72.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.72.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.72.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.130.keytab.
Entry for principal oryx/127.23.72.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.130.keytab.
WARNING: no policy specified for oryx/127.23.72.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.72.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.72.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.131.keytab.
Entry for principal oryx/127.23.72.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.131.keytab.
I20260808 18:23:09.757977 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 416
I20260808 18:23:09.765017 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 562
I20260808 18:23:09.771890 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 706
I20260808 18:23:09.781728 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 315
I20260808 18:23:09.790086 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:41351
--webserver_interface=127.23.72.190
--webserver_port=41085
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:35157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:41351
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:23:09.900784 855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:23:09.901023 855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:23:09.901060 855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:23:09.905001 855 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:23:09.905049 855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:23:09.905068 855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:23:09.905121 855 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:23:09.905157 855 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:23:09.910815 855 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:35157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:41351
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:41351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.72.190
--webserver_port=41085
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.855
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:23:09.912139 855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:23:09.913364 855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:23:09.921873 865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:23:09.921947 855 server_base.cc:1034] running on GCE node
W20260808 18:23:09.921994 860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:23:09.922151 861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:23:09.922717 855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:23:09.923897 855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:23:09.925082 855 hybrid_clock.cc:648] HybridClock initialized: now 1786213389925051 us; error 45 us; skew 500 ppm
Aug 08 18:23:09 dist-test-slave-5k54 krb5kdc[301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213389, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:23:09.927832 855 init.cc:377] Logged in from keytab as kudu/127.23.72.190@KRBTEST.COM (short username kudu)
I20260808 18:23:09.928668 855 webserver.cc:460] Webserver started at http://127.23.72.190:41085/ using document root <none> and password file <none>
I20260808 18:23:09.929219 855 fs_manager.cc:359] Metadata directory not provided
I20260808 18:23:09.929335 855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:23:09.933146 855 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 18:23:09.935181 872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:09.936415 855 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:23:09.936566 855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "a986ee0f4ea045c48dc6682a417e9a51"
format_stamp: "Formatted at 2026-08-08 18:23:08 on dist-test-slave-5k54"
server_key: "d1130aaaeb1238012be2c0bef85d49b0"
server_key_iv: "a6e7488fc65593049824dab92e274981"
server_key_version: "encryptionkey@0"
I20260808 18:23:09.937109 855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:23:09.951889 855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:23:09.954627 855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:23:09.954824 855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:23:09.961999 855 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.190:41351
I20260808 18:23:09.962018 929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.190:41351 every 8 connection(s)
I20260808 18:23:09.963071 855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:23:09.966683 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 855
I20260808 18:23:09.967161 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.129:43481
--local_ip_for_outbound_sockets=127.23.72.129
--tserver_master_addrs=127.23.72.190:41351
--webserver_port=42349
--webserver_interface=127.23.72.129
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab
--principal=kudu/127.23.72.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:35157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.23.72.129
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:23:09.967710 930 sys_catalog.cc:263] Verifying existing consensus state
I20260808 18:23:09.969893 930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a986ee0f4ea045c48dc6682a417e9a51: Bootstrap starting.
I20260808 18:23:09.987002 930 log.cc:824] T 00000000000000000000000000000000 P a986ee0f4ea045c48dc6682a417e9a51: Log is configured to *not* fsync() on all Append() calls
I20260808 18:23:09.993355 930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a986ee0f4ea045c48dc6682a417e9a51: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 18:23:09.993727 930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a986ee0f4ea045c48dc6682a417e9a51: Bootstrap complete.
I20260808 18:23:09.997927 930 raft_consensus.cc:348] T 00000000000000000000000000000000 P a986ee0f4ea045c48dc6682a417e9a51 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a986ee0f4ea045c48dc6682a417e9a51" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 41351 } }
I20260808 18:23:09.998302 930 raft_consensus.cc:729] T 00000000000000000000000000000000 P a986ee0f4ea045c48dc6682a417e9a51 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a986ee0f4ea045c48dc6682a417e9a51, State: Initialized, Role: FOLLOWER
I20260808 18:23:09.998812 930 consensus_queue.cc:260] T 00000000000000000000000000000000 P a986ee0f4ea045c48dc6682a417e9a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a986ee0f4ea045c48dc6682a417e9a51" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 41351 } }
I20260808 18:23:09.998935 930 raft_consensus.cc:388] T 00000000000000000000000000000000 P a986ee0f4ea045c48dc6682a417e9a51 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:23:09.999022 930 raft_consensus.cc:482] T 00000000000000000000000000000000 P a986ee0f4ea045c48dc6682a417e9a51 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:23:09.999119 930 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a986ee0f4ea045c48dc6682a417e9a51 [term 1 FOLLOWER]: Advancing to term 2
I20260808 18:23:10.001029 930 raft_consensus.cc:504] T 00000000000000000000000000000000 P a986ee0f4ea045c48dc6682a417e9a51 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a986ee0f4ea045c48dc6682a417e9a51" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 41351 } }
I20260808 18:23:10.001355 930 leader_election.cc:302] T 00000000000000000000000000000000 P a986ee0f4ea045c48dc6682a417e9a51 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a986ee0f4ea045c48dc6682a417e9a51; no voters:
I20260808 18:23:10.001677 930 leader_election.cc:288] T 00000000000000000000000000000000 P a986ee0f4ea045c48dc6682a417e9a51 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 18:23:10.001771 936 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a986ee0f4ea045c48dc6682a417e9a51 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 18:23:10.001987 936 raft_consensus.cc:686] T 00000000000000000000000000000000 P a986ee0f4ea045c48dc6682a417e9a51 [term 2 LEADER]: Becoming Leader. State: Replica: a986ee0f4ea045c48dc6682a417e9a51, State: Running, Role: LEADER
I20260808 18:23:10.002368 936 consensus_queue.cc:237] T 00000000000000000000000000000000 P a986ee0f4ea045c48dc6682a417e9a51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a986ee0f4ea045c48dc6682a417e9a51" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 41351 } }
I20260808 18:23:10.002967 930 sys_catalog.cc:564] T 00000000000000000000000000000000 P a986ee0f4ea045c48dc6682a417e9a51 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:23:10.003332 936 sys_catalog.cc:455] T 00000000000000000000000000000000 P a986ee0f4ea045c48dc6682a417e9a51 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a986ee0f4ea045c48dc6682a417e9a51. Latest consensus state: current_term: 2 leader_uuid: "a986ee0f4ea045c48dc6682a417e9a51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a986ee0f4ea045c48dc6682a417e9a51" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 41351 } } }
I20260808 18:23:10.003471 936 sys_catalog.cc:458] T 00000000000000000000000000000000 P a986ee0f4ea045c48dc6682a417e9a51 [sys.catalog]: This master's current role is: LEADER
I20260808 18:23:10.003921 940 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:23:10.003871 937 sys_catalog.cc:455] T 00000000000000000000000000000000 P a986ee0f4ea045c48dc6682a417e9a51 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a986ee0f4ea045c48dc6682a417e9a51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a986ee0f4ea045c48dc6682a417e9a51" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 41351 } } }
I20260808 18:23:10.004103 937 sys_catalog.cc:458] T 00000000000000000000000000000000 P a986ee0f4ea045c48dc6682a417e9a51 [sys.catalog]: This master's current role is: LEADER
I20260808 18:23:10.007262 940 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:23:10.007923 940 catalog_manager.cc:1259] Loaded cluster ID: a32ad68aaf8c46a9829951fc0c45be96
I20260808 18:23:10.007987 940 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:23:10.010303 940 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:23:10.010954 940 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a986ee0f4ea045c48dc6682a417e9a51: Loaded TSK: 0
I20260808 18:23:10.012197 940 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 18:23:10.082320 933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:23:10.082602 933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:23:10.082675 933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:23:10.086826 933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:23:10.086927 933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:23:10.087019 933 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.129
I20260808 18:23:10.092303 933 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:35157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.129.keytab
--principal=oryx/127.23.72.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.129:43481
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.72.129
--webserver_port=42349
--tserver_master_addrs=127.23.72.190:41351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.933
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.129
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:23:10.093282 933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:23:10.094302 933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:23:10.104835 962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:23:10.105022 933 server_base.cc:1034] running on GCE node
W20260808 18:23:10.104837 960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:23:10.105159 964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:23:10.105638 933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:23:10.106300 933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:23:10.107502 933 hybrid_clock.cc:648] HybridClock initialized: now 1786213390107485 us; error 36 us; skew 500 ppm
Aug 08 18:23:10 dist-test-slave-5k54 krb5kdc[301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213390, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.72.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:23:10.110613 933 init.cc:377] Logged in from keytab as oryx/127.23.72.129@KRBTEST.COM (short username oryx)
I20260808 18:23:10.111763 933 webserver.cc:460] Webserver started at http://127.23.72.129:42349/ using document root <none> and password file <none>
I20260808 18:23:10.112495 933 fs_manager.cc:359] Metadata directory not provided
I20260808 18:23:10.112607 933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:23:10.117831 933 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 18:23:10.120414 973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:10.121963 933 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:23:10.122097 933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-0/wal
uuid: "7861831876c84f38af0aaf70843c598a"
format_stamp: "Formatted at 2026-08-08 18:23:09 on dist-test-slave-5k54"
server_key: "ecd211fc4224cd1b0abc370893641188"
server_key_iv: "4516236220cdcf93cbd9bd2f4674b794"
server_key_version: "encryptionkey@0"
I20260808 18:23:10.122812 933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:23:10.135842 933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:23:10.139142 933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:23:10.139344 933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:23:10.140081 933 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:23:10.140973 933 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:23:10.141023 933 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:10.141088 933 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:23:10.141193 933 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:10.151165 933 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.129:43481
I20260808 18:23:10.151228 1096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.129:43481 every 8 connection(s)
I20260808 18:23:10.152240 933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
I20260808 18:23:10.154779 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 933
I20260808 18:23:10.155295 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.130:32811
--local_ip_for_outbound_sockets=127.23.72.130
--tserver_master_addrs=127.23.72.190:41351
--webserver_port=39473
--webserver_interface=127.23.72.130
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab
--principal=kudu/127.23.72.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:35157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.23.72.130
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 18:23:10 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.129@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:10 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.129@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 18:23:10.167413 1099 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:10.154230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:33637 (local address 127.23.72.190:41351)
0808 18:23:10.154676 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:10.154695 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:10.155055 (+ 360us) server_negotiation.cc:407] Connection header received
0808 18:23:10.156960 (+ 1905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:10.156971 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:10.157250 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:10.157586 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:10.158986 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:10.160050 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:10.161988 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:10.162452 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:10.162576 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:10.166665 (+ 4089us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.129:33637: BlockingRecv error: recv got EOF from 127.23.72.129:33637 (error 108)
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":134,"threads_started":1}
I20260808 18:23:10.167451 1100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:10.154324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.129:33637)
0808 18:23:10.154869 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:10.154934 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:10.156508 (+ 1574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:10.157666 (+ 1158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:10.157673 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:10.158108 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:10.158793 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:10.158816 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:10.160196 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:10.160203 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:10.161843 (+ 1640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:10.161858 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:10.162651 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:10.162658 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:10.162899 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:10.163631 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:10.163659 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:10.166424 (+ 2765us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":169,"threads_started":1}
W20260808 18:23:10.168286 1097 heartbeater.cc:644] Failed to heartbeat to 127.23.72.190:41351 (0 consecutive failures): Not authorized: Failed to ping master at 127.23.72.190:41351: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Aug 08 18:23:10 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.129@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:10 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.129@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:10.174268 1100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:10.168776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.129:40699)
0808 18:23:10.169021 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:10.169036 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:10.169124 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:10.169384 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:10.169387 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:10.169559 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:10.169648 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:10.169653 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:10.170656 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:10.170662 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:10.171216 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:10.171224 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:10.171766 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:10.171770 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:10.171864 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:10.172355 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:10.172376 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:10.174121 (+ 1745us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178}
W20260808 18:23:10.174361 1099 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:10.168860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:40699 (local address 127.23.72.190:41351)
0808 18:23:10.169012 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:10.169016 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:10.169081 (+ 65us) server_negotiation.cc:407] Connection header received
0808 18:23:10.169227 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:10.169230 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:10.169281 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:10.169367 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:10.169761 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:10.170518 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:10.171347 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:10.171641 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:10.171750 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:10.174247 (+ 2497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.129:40699: BlockingRecv error: recv got EOF from 127.23.72.129:40699 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 08 18:23:10 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.129@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:10 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.129@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:10.180012 1100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:10.174978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.129:32821)
0808 18:23:10.175232 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:10.175245 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:10.175332 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:10.175593 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:10.175596 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:10.175751 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:10.175819 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:10.175823 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:10.176646 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:10.176651 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:10.177190 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:10.177198 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:10.177648 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:10.177652 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:10.177728 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:10.178208 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:10.178228 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:10.179868 (+ 1640us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187}
W20260808 18:23:10.180096 1099 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:10.175098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:32821 (local address 127.23.72.190:41351)
0808 18:23:10.175210 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:10.175214 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:10.175306 (+ 92us) server_negotiation.cc:407] Connection header received
0808 18:23:10.175405 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:10.175408 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:10.175492 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:10.175578 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:10.175932 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:10.176504 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:10.177317 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:10.177541 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:10.177648 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:10.179982 (+ 2334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.129:32821: BlockingRecv error: recv got EOF from 127.23.72.129:32821 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260808 18:23:10.180439 1097 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 18:23:10.270059 1102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:23:10.270340 1102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:23:10.270412 1102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:23:10.274401 1102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:23:10.274483 1102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:23:10.274567 1102 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.130
I20260808 18:23:10.279728 1102 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:35157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.130.keytab
--principal=oryx/127.23.72.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.130:32811
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.72.130
--webserver_port=39473
--tserver_master_addrs=127.23.72.190:41351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1102
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.130
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:23:10.280719 1102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:23:10.281728 1102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:23:10.291045 1108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:23:10.291091 1109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:23:10.291253 1102 server_base.cc:1034] running on GCE node
W20260808 18:23:10.291229 1111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:23:10.291798 1102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:23:10.292449 1102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:23:10.293641 1102 hybrid_clock.cc:648] HybridClock initialized: now 1786213390293611 us; error 41 us; skew 500 ppm
Aug 08 18:23:10 dist-test-slave-5k54 krb5kdc[301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213390, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.72.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:23:10.296350 1102 init.cc:377] Logged in from keytab as oryx/127.23.72.130@KRBTEST.COM (short username oryx)
I20260808 18:23:10.297197 1102 webserver.cc:460] Webserver started at http://127.23.72.130:39473/ using document root <none> and password file <none>
I20260808 18:23:10.297753 1102 fs_manager.cc:359] Metadata directory not provided
I20260808 18:23:10.297863 1102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:23:10.301780 1102 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:23:10.303946 1119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:10.305214 1102 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:23:10.305305 1102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-1/wal
uuid: "6b90e1b6cb494402b025b337650c8ca7"
format_stamp: "Formatted at 2026-08-08 18:23:09 on dist-test-slave-5k54"
server_key: "76e0eb51edf8522e3f9f283aa6df1f8b"
server_key_iv: "e9c481ba657a5ff9e44c76e1a6b53901"
server_key_version: "encryptionkey@0"
I20260808 18:23:10.305790 1102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:23:10.337666 1102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:23:10.340768 1102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:23:10.340930 1102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:23:10.341466 1102 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:23:10.342314 1102 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:23:10.342360 1102 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:10.342399 1102 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:23:10.342414 1102 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:10.351255 1102 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.130:32811
I20260808 18:23:10.351285 1237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.130:32811 every 8 connection(s)
I20260808 18:23:10.352298 1102 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
I20260808 18:23:10.352905 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 1102
I20260808 18:23:10.353430 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.131:36489
--local_ip_for_outbound_sockets=127.23.72.131
--tserver_master_addrs=127.23.72.190:41351
--webserver_port=34445
--webserver_interface=127.23.72.131
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab
--principal=kudu/127.23.72.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:35157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.23.72.131
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 18:23:10 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.130@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:10 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.130@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 18:23:10.365525 1099 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:10.354242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:57957 (local address 127.23.72.190:41351)
0808 18:23:10.354420 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:10.354427 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:10.355276 (+ 849us) server_negotiation.cc:407] Connection header received
0808 18:23:10.356803 (+ 1527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:10.356805 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:10.356857 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:10.356935 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:10.358538 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:10.359054 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:10.360970 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:10.361184 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:10.361264 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:10.365425 (+ 4161us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.130:57957: BlockingRecv error: recv got EOF from 127.23.72.130:57957 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 18:23:10.366371 1240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:10.354590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.130:57957)
0808 18:23:10.355102 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:10.355168 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:10.356581 (+ 1413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:10.357125 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:10.357135 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:10.357658 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:10.358374 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:10.358397 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:10.359184 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:10.359191 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:10.360808 (+ 1617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:10.360823 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:10.361345 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:10.361350 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:10.361587 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:10.362209 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:10.362237 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:10.365304 (+ 3067us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":113,"threads_started":1}
W20260808 18:23:10.366847 1238 heartbeater.cc:644] Failed to heartbeat to 127.23.72.190:41351 (0 consecutive failures): Not authorized: Failed to ping master at 127.23.72.190:41351: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Aug 08 18:23:10 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.130@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:10 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.130@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:10.372524 1240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:10.367244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.130:56239)
0808 18:23:10.367434 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:10.367479 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:10.367538 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:10.367768 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:10.367770 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:10.367891 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:10.367996 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:10.368000 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:10.368988 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:10.368991 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:10.369516 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:10.369524 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:10.370063 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:10.370065 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:10.370129 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:10.370584 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:10.370596 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:10.372356 (+ 1760us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128}
W20260808 18:23:10.372588 1099 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:10.367370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:56239 (local address 127.23.72.190:41351)
0808 18:23:10.367508 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:10.367512 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:10.367526 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:23:10.367617 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:10.367620 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:10.367663 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:10.367747 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:10.368093 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:10.368871 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:10.369616 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:10.369961 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:10.370057 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:10.372484 (+ 2427us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.130:56239: BlockingRecv error: recv got EOF from 127.23.72.130:56239 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 08 18:23:10 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.130@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:10 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.130@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:10.377935 1240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:10.373116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.130:36511)
0808 18:23:10.373294 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:10.373305 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:10.373360 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:10.373569 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:10.373570 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:10.373686 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:10.373747 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:10.373751 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:10.374703 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:10.374705 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:10.375207 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:10.375214 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:10.375768 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:10.375770 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:10.375826 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:10.376278 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:10.376290 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:10.377849 (+ 1559us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110}
W20260808 18:23:10.378109 1238 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 18:23:10.378283 1099 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:10.373217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:36511 (local address 127.23.72.190:41351)
0808 18:23:10.373331 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:10.373335 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:10.373348 (+ 13us) server_negotiation.cc:407] Connection header received
0808 18:23:10.373434 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:10.373437 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:10.373474 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:10.373546 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:10.373843 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:10.374601 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:10.375313 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:10.375669 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:10.375770 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:10.377998 (+ 2228us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.130:36511: BlockingRecv error: recv got EOF from 127.23.72.130:36511 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260808 18:23:10.469527 1241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:23:10.469763 1241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:23:10.469805 1241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:23:10.473802 1241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:23:10.473853 1241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:23:10.473908 1241 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.131
I20260808 18:23:10.478894 1241 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:35157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/krb5kdc/oryx_127.23.72.131.keytab
--principal=oryx/127.23.72.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.131:36489
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.72.131
--webserver_port=34445
--tserver_master_addrs=127.23.72.190:41351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1241
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.131
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:23:10.479873 1241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:23:10.480844 1241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:23:10.490501 1248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:23:10.490475 1250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:23:10.490482 1246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:23:10.490793 1241 server_base.cc:1034] running on GCE node
I20260808 18:23:10.491155 1241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:23:10.491760 1241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:23:10.492920 1241 hybrid_clock.cc:648] HybridClock initialized: now 1786213390492902 us; error 30 us; skew 500 ppm
Aug 08 18:23:10 dist-test-slave-5k54 krb5kdc[301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213390, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.72.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:23:10.495743 1241 init.cc:377] Logged in from keytab as oryx/127.23.72.131@KRBTEST.COM (short username oryx)
I20260808 18:23:10.496592 1241 webserver.cc:460] Webserver started at http://127.23.72.131:34445/ using document root <none> and password file <none>
I20260808 18:23:10.497123 1241 fs_manager.cc:359] Metadata directory not provided
I20260808 18:23:10.497216 1241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:23:10.500964 1241 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:23:10.502995 1258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:10.504453 1241 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 18:23:10.504575 1241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-2/wal
uuid: "46f0bc7d12164a70aa36f32f68fba096"
format_stamp: "Formatted at 2026-08-08 18:23:09 on dist-test-slave-5k54"
server_key: "7cf97126c42ecf276eb12ca79c897745"
server_key_iv: "ca6bc0c3e6c1f0ac4fa4933a7aea02c9"
server_key_version: "encryptionkey@0"
I20260808 18:23:10.505101 1241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:23:10.521874 1241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:23:10.524657 1241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:23:10.524818 1241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:23:10.525321 1241 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:23:10.526067 1241 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:23:10.526111 1241 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:10.526150 1241 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:23:10.526171 1241 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:10.535053 1241 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.131:36489
I20260808 18:23:10.535082 1380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.131:36489 every 8 connection(s)
I20260808 18:23:10.536039 1241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
I20260808 18:23:10.541006 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 1241
Aug 08 18:23:10 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.131@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:10 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.131@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 18:23:10.547580 1099 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:10.537701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:46937 (local address 127.23.72.190:41351)
0808 18:23:10.537831 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:10.537834 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:10.538623 (+ 789us) server_negotiation.cc:407] Connection header received
0808 18:23:10.540013 (+ 1390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:10.540016 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:10.540074 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:10.540147 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:10.541482 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:10.542229 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:10.543409 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:10.543707 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:10.543822 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:10.547460 (+ 3638us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.131:46937: BlockingRecv error: recv got EOF from 127.23.72.131:46937 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 18:23:10.548290 1383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:10.538063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.131:46937)
0808 18:23:10.538487 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:10.538527 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:10.539801 (+ 1274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:10.540312 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:10.540319 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:10.540659 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:10.541315 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:10.541334 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:10.542364 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:10.542368 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:10.543279 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:10.543288 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:10.543859 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:10.543864 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:10.544036 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:10.544588 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:10.544604 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:10.547344 (+ 2740us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":105,"threads_started":1}
W20260808 18:23:10.548770 1381 heartbeater.cc:644] Failed to heartbeat to 127.23.72.190:41351 (0 consecutive failures): Not authorized: Failed to ping master at 127.23.72.190:41351: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
I20260808 18:23:10.551945 1385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:10.542044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43058 (local address 127.23.72.190:41351)
0808 18:23:10.542292 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:10.542296 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:10.542414 (+ 118us) server_negotiation.cc:407] Connection header received
0808 18:23:10.542571 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:10.542575 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:10.542623 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:10.542716 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:10.543475 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:10.544069 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:10.545068 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:10.545431 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:10.545545 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:10.547283 (+ 1738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:23:10.547289 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:23:10.547301 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:23:10.547326 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:23:10.549936 (+ 2610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:23:10.550328 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:23:10.550333 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:23:10.550339 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:23:10.550485 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:23:10.550766 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:23:10.550768 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:23:10.550769 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:23:10.551085 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:23:10.551225 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:10.551557 (+ 332us) server_negotiation.cc:299] Negotiation successful
0808 18:23:10.551770 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":117,"threads_started":1}
Aug 08 18:23:10 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.131@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:10 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.131@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:10.554385 1383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:10.549226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.131:45139)
0808 18:23:10.549332 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:10.549359 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:10.549428 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:10.549633 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:10.549635 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:10.549754 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:10.549833 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:10.549837 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:10.550665 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:10.550669 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:10.551301 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:10.551311 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:10.551847 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:10.551851 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:10.551919 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:10.552385 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:10.552402 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:10.554263 (+ 1861us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 18:23:10.554507 1099 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:10.549300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:45139 (local address 127.23.72.190:41351)
0808 18:23:10.549410 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:10.549414 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:10.549428 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:23:10.549489 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:10.549492 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:10.549530 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:10.549616 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:10.549938 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:10.550567 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:10.551445 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:10.551736 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:10.551842 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:10.554415 (+ 2573us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.131:45139: BlockingRecv error: recv got EOF from 127.23.72.131:45139 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 08 18:23:10 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.131@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:10 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.131@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:10.559216 1383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:10.555130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.131:51991)
0808 18:23:10.555253 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:10.555268 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:10.555341 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:10.555570 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:10.555572 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:10.555690 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:10.555754 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:10.555758 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:10.556367 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:10.556369 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:10.556868 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:10.556875 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:10.557234 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:10.557236 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:10.557288 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:10.557634 (+ 346us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:10.557644 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:10.559138 (+ 1494us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 18:23:10.559399 1099 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:10.555212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:51991 (local address 127.23.72.190:41351)
0808 18:23:10.555332 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:10.555335 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:10.555352 (+ 17us) server_negotiation.cc:407] Connection header received
0808 18:23:10.555411 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:10.555413 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:10.555490 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:10.555556 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:10.555843 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:10.556279 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:10.556961 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:10.557147 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:10.557218 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:10.559245 (+ 2027us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.131:51991: BlockingRecv error: recv got EOF from 127.23.72.131:51991 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260808 18:23:10.559463 1381 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 08 18:23:11 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.129@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:11 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.129@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:11.186288 1386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:11.181071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.129:37265)
0808 18:23:11.181358 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:11.181372 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:11.181468 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:11.181707 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:11.181709 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:11.181885 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:11.181993 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:11.181998 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:11.182715 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:11.182717 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:11.183282 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:11.183289 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:11.183718 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:11.183721 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:11.183795 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:11.184277 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:11.184288 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:11.186166 (+ 1878us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":72,"threads_started":1}
W20260808 18:23:11.186470 1388 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:11.181129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:37265 (local address 127.23.72.190:41351)
0808 18:23:11.181417 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:11.181420 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:11.181433 (+ 13us) server_negotiation.cc:407] Connection header received
0808 18:23:11.181553 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:11.181555 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:11.181599 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:11.181675 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:11.182091 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:11.182628 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:11.183390 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:11.183625 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:11.183700 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:11.186307 (+ 2607us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.129:37265: BlockingRecv error: recv got EOF from 127.23.72.129:37265 (error 108)
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":109,"threads_started":1}
Aug 08 18:23:11 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.130@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:11 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.130@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:11.385010 1389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:11.378758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.130:53761)
0808 18:23:11.379128 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:11.379145 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:11.379262 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:11.379584 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:11.379587 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:11.379812 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:11.379905 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:11.379910 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:11.380683 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:11.380688 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:11.381269 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:11.381277 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:11.381764 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:11.381768 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:11.381869 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:11.382471 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:11.382500 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:11.384878 (+ 2378us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":133,"threads_started":1}
W20260808 18:23:11.385141 1388 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:11.378961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:53761 (local address 127.23.72.190:41351)
0808 18:23:11.379140 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:11.379146 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:11.379206 (+ 60us) server_negotiation.cc:407] Connection header received
0808 18:23:11.379380 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:11.379383 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:11.379473 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:11.379562 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:11.380029 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:11.380567 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:11.381395 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:11.381645 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:11.381757 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:11.385022 (+ 3265us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.130:53761: BlockingRecv error: recv got EOF from 127.23.72.130:53761 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
Aug 08 18:23:11 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.131@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:11 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.131@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:11.565543 1390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:11.560041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.131:43001)
0808 18:23:11.560316 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:11.560329 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:11.560427 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:11.560686 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:11.560688 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:11.560901 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:11.561003 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:11.561009 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:11.561733 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:11.561736 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:11.562344 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:11.562353 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:11.562773 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:11.562775 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:11.562836 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:11.563481 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:11.563493 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:11.565422 (+ 1929us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":100,"threads_started":1}
W20260808 18:23:11.565681 1388 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:11.560121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:43001 (local address 127.23.72.190:41351)
0808 18:23:11.560252 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:11.560256 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:11.560407 (+ 151us) server_negotiation.cc:407] Connection header received
0808 18:23:11.560504 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:11.560506 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:11.560552 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:11.560623 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:11.561109 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:11.561640 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:11.562445 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:11.562649 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:11.562754 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:11.565562 (+ 2808us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.131:43001: BlockingRecv error: recv got EOF from 127.23.72.131:43001 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 08 18:23:12 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.129@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:12 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.129@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:12.193084 1391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:12.187172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.129:35227)
0808 18:23:12.187504 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:12.187518 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:12.187610 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:12.187856 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:12.187858 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:12.188059 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:12.188158 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:12.188164 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:12.189141 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:12.189144 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:12.189945 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:12.189955 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:12.190376 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:12.190378 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:12.190443 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:12.191048 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:12.191061 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:12.192946 (+ 1885us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":87,"threads_started":1}
W20260808 18:23:12.193248 1392 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:12.187272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:35227 (local address 127.23.72.190:41351)
0808 18:23:12.187562 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:12.187565 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:12.187579 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:23:12.187697 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:12.187699 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:12.187744 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:12.187821 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:12.188253 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:12.189028 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:12.190050 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:12.190279 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:12.190366 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:12.193079 (+ 2713us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.129:35227: BlockingRecv error: recv got EOF from 127.23.72.129:35227 (error 108)
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":82,"threads_started":1}
Aug 08 18:23:12 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.130@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:12 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.130@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:12.391235 1393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:12.385948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.130:54649)
0808 18:23:12.386207 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:12.386221 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:12.386316 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:12.386559 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:12.386561 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:12.386740 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:12.386836 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:12.386840 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:12.387661 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:12.387664 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:12.388218 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:12.388225 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:12.388630 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:12.388633 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:12.388694 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:12.389187 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:12.389199 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:12.391121 (+ 1922us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":82,"threads_started":1}
W20260808 18:23:12.391381 1392 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:12.385998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:54649 (local address 127.23.72.190:41351)
0808 18:23:12.386173 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:12.386178 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:12.386305 (+ 127us) server_negotiation.cc:407] Connection header received
0808 18:23:12.386396 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:12.386398 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:12.386443 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:12.386525 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:12.386956 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:12.387558 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:12.388315 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:12.388532 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:12.388607 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:12.391264 (+ 2657us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.130:54649: BlockingRecv error: recv got EOF from 127.23.72.130:54649 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 08 18:23:12 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.131@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:12 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.131@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:12.576916 1394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:12.566510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.131:38523)
0808 18:23:12.566830 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:12.566849 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:12.566976 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:12.567271 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:12.567281 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:12.567568 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:12.567666 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:12.567672 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:12.568549 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:12.568554 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:12.569175 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:12.569185 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:12.569653 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:12.569656 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:12.569738 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:12.574748 (+ 5010us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:12.574760 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:12.576682 (+ 1922us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":90,"threads_started":1}
W20260808 18:23:12.576939 1392 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:12.566655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:38523 (local address 127.23.72.190:41351)
0808 18:23:12.566864 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:12.566872 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:12.566912 (+ 40us) server_negotiation.cc:407] Connection header received
0808 18:23:12.567087 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:12.567091 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:12.567146 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:12.567247 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:12.567803 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:12.568427 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:12.569309 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:12.569538 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:12.569649 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:12.576828 (+ 7179us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.131:38523: BlockingRecv error: recv got EOF from 127.23.72.131:38523 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 08 18:23:13 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.129@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:13 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.129@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:13.199803 1395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:13.193945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.129:54775)
0808 18:23:13.194261 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:13.194294 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:13.194415 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:13.194686 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:13.194689 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:13.194911 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:13.195019 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:13.195025 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:13.195896 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:13.195899 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:13.196646 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:13.196657 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:13.197139 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:13.197142 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:13.197210 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:13.197774 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:13.197785 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:13.199682 (+ 1897us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":122,"threads_started":1}
W20260808 18:23:13.199965 1396 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:13.194029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:54775 (local address 127.23.72.190:41351)
0808 18:23:13.194269 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:13.194273 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:13.194399 (+ 126us) server_negotiation.cc:407] Connection header received
0808 18:23:13.194499 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:13.194501 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:13.194545 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:13.194624 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:13.195186 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:13.195759 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:13.196762 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:13.197006 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:13.197102 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:13.199826 (+ 2724us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.129:54775: BlockingRecv error: recv got EOF from 127.23.72.129:54775 (error 108)
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":89,"threads_started":1}
Aug 08 18:23:13 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.130@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:13 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.130@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:13.397449 1397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:13.392025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.130:37387)
0808 18:23:13.392295 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:13.392309 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:13.392435 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:13.392651 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:13.392653 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:13.392837 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:13.392940 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:13.392945 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:13.393678 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:13.393680 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:13.394239 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:13.394247 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:13.394698 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:13.394700 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:13.394763 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:13.395261 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:13.395273 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:13.397314 (+ 2041us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":71,"threads_started":1}
W20260808 18:23:13.397632 1396 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:13.392137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:37387 (local address 127.23.72.190:41351)
0808 18:23:13.392318 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:13.392321 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:13.392390 (+ 69us) server_negotiation.cc:407] Connection header received
0808 18:23:13.392503 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:13.392505 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:13.392546 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:13.392613 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:13.393043 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:13.393582 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:13.394373 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:13.394593 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:13.394682 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:13.397472 (+ 2790us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.130:37387: BlockingRecv error: recv got EOF from 127.23.72.130:37387 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 08 18:23:13 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.131@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:13 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.131@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:13.583352 1398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:13.577743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.131:52851)
0808 18:23:13.578073 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:13.578085 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:13.578233 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:13.578512 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:13.578515 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:13.578721 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:13.578814 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:13.578820 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:13.579661 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:13.579667 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:13.580226 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:13.580234 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:13.580716 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:13.580719 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:13.580810 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:13.581354 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:13.581365 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:13.583184 (+ 1819us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":128,"threads_started":1}
W20260808 18:23:13.583456 1396 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:13.577800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:52851 (local address 127.23.72.190:41351)
0808 18:23:13.577968 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:13.577973 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:13.578159 (+ 186us) server_negotiation.cc:407] Connection header received
0808 18:23:13.578334 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:13.578338 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:13.578389 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:13.578484 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:13.578937 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:13.579554 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:13.580349 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:13.580589 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:13.580699 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:13.583314 (+ 2615us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.131:52851: BlockingRecv error: recv got EOF from 127.23.72.131:52851 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 08 18:23:14 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.129@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:14 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.129@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:14.206511 1399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:14.200621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.129:50831)
0808 18:23:14.200902 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:14.200915 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:14.201002 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:14.201237 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:14.201239 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:14.201427 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:14.201537 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:14.201542 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:14.202257 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:14.202259 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:14.202806 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:14.202813 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:14.203380 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:14.203384 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:14.203496 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:14.204054 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:14.204067 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:14.206399 (+ 2332us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":90,"threads_started":1}
W20260808 18:23:14.206671 1401 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:14.200681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:50831 (local address 127.23.72.190:41351)
0808 18:23:14.200967 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:14.200971 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:14.200984 (+ 13us) server_negotiation.cc:407] Connection header received
0808 18:23:14.201089 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:14.201091 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:14.201135 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:14.201203 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:14.201640 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:14.202166 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:14.202935 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:14.203261 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:14.203371 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:14.206538 (+ 3167us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.129:50831: BlockingRecv error: recv got EOF from 127.23.72.129:50831 (error 108)
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":95,"threads_started":1}
Aug 08 18:23:14 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.130@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:14 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.130@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:14.404163 1402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:14.398191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.130:34089)
0808 18:23:14.398539 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:14.398551 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:14.398687 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:14.398960 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:14.398963 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:14.399178 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:14.399289 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:14.399295 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:14.400159 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:14.400165 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:14.400753 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:14.400762 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:14.401320 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:14.401324 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:14.401406 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:14.401967 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:14.401980 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:14.403996 (+ 2016us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":88,"threads_started":1}
W20260808 18:23:14.404436 1401 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:14.398312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:34089 (local address 127.23.72.190:41351)
0808 18:23:14.398485 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:14.398490 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:14.398627 (+ 137us) server_negotiation.cc:407] Connection header received
0808 18:23:14.398788 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:14.398792 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:14.398846 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:14.398938 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:14.399463 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:14.400032 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:14.400910 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:14.401195 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:14.401315 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:14.404178 (+ 2863us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.130:34089: BlockingRecv error: recv got EOF from 127.23.72.130:34089 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 08 18:23:14 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.131@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:14 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.131@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:14.589809 1403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:14.584227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.131:50267)
0808 18:23:14.584542 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:14.584555 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:14.584662 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:14.584960 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:14.584963 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:14.585145 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:14.585241 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:14.585246 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:14.586062 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:14.586066 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:14.586631 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:14.586640 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:14.587137 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:14.587140 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:14.587229 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:14.587806 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:14.587818 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:14.589639 (+ 1821us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":98,"threads_started":1}
W20260808 18:23:14.589900 1401 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:14.584280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:50267 (local address 127.23.72.190:41351)
0808 18:23:14.584449 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:14.584454 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:14.584647 (+ 193us) server_negotiation.cc:407] Connection header received
0808 18:23:14.584731 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:14.584733 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:14.584795 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:14.584906 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:14.585368 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:14.585916 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:14.586757 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:14.587012 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:14.587128 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:14.589759 (+ 2631us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.131:50267: BlockingRecv error: recv got EOF from 127.23.72.131:50267 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 08 18:23:15 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.129@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:15 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.129@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:15.213438 1404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:15.207436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.129:43329)
0808 18:23:15.207751 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:15.207764 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:15.207866 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:15.208118 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:15.208120 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:15.208362 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:15.208535 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:15.208544 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:15.209637 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:15.209643 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:15.210310 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:15.210321 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:15.210759 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:15.210761 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:15.210826 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:15.211334 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:15.211345 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:15.213242 (+ 1897us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":91,"threads_started":1}
W20260808 18:23:15.213496 1405 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:15.207522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:43329 (local address 127.23.72.190:41351)
0808 18:23:15.207765 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:15.207768 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:15.207840 (+ 72us) server_negotiation.cc:407] Connection header received
0808 18:23:15.207954 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:15.207956 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:15.208001 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:15.208082 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:15.208661 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:15.209493 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:15.210431 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:15.210651 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:15.210733 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:15.213370 (+ 2637us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.129:43329: BlockingRecv error: recv got EOF from 127.23.72.129:43329 (error 108)
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":63,"threads_started":1}
Aug 08 18:23:15 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.130@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:15 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.130@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:15.410444 1406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:15.405026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.130:59505)
0808 18:23:15.405321 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:15.405354 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:15.405487 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:15.405739 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:15.405741 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:15.405934 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:15.406027 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:15.406033 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:15.406735 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:15.406737 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:15.407305 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:15.407312 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:15.407735 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:15.407738 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:15.407803 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:15.408311 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:15.408322 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:15.410330 (+ 2008us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":87,"threads_started":1}
W20260808 18:23:15.410626 1405 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:15.405182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:59505 (local address 127.23.72.190:41351)
0808 18:23:15.405329 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:15.405336 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:15.405457 (+ 121us) server_negotiation.cc:407] Connection header received
0808 18:23:15.405589 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:15.405591 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:15.405635 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:15.405703 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:15.406129 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:15.406655 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:15.407406 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:15.407639 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:15.407715 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:15.410466 (+ 2751us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.130:59505: BlockingRecv error: recv got EOF from 127.23.72.130:59505 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 08 18:23:15 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.131@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:15 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.131@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:15.596215 1408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:15.590633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.131:42625)
0808 18:23:15.590951 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:15.590964 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:15.591092 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:15.591365 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:15.591368 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:15.591619 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:15.591712 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:15.591718 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:15.592483 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:15.592488 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:15.593052 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:15.593060 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:15.593530 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:15.593534 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:15.593620 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:15.594147 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:15.594158 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:15.596047 (+ 1889us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":77,"threads_started":1}
W20260808 18:23:15.596280 1405 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:15.590687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:42625 (local address 127.23.72.190:41351)
0808 18:23:15.590847 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:15.590852 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:15.591040 (+ 188us) server_negotiation.cc:407] Connection header received
0808 18:23:15.591193 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:15.591196 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:15.591251 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:15.591345 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:15.591830 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:15.592373 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:15.593173 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:15.593416 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:15.593527 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:15.596171 (+ 2644us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.131:42625: BlockingRecv error: recv got EOF from 127.23.72.131:42625 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 08 18:23:16 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.129@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:16 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.129@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:16.219961 1409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:16.214397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.129:60593)
0808 18:23:16.214685 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:16.214704 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:16.214802 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:16.215031 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:16.215033 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:16.215227 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:16.215327 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:16.215331 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:16.216144 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:16.216147 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:16.216719 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:16.216727 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:16.217208 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:16.217211 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:16.217276 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:16.217822 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:16.217835 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:16.219790 (+ 1955us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":70,"threads_started":1}
W20260808 18:23:16.220070 1410 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:16.214493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:60593 (local address 127.23.72.190:41351)
0808 18:23:16.214809 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:16.214812 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:16.214827 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:23:16.214867 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:16.214869 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:16.214912 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:16.214996 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:16.215481 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:16.216031 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:16.216864 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:16.217102 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:16.217187 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:16.219934 (+ 2747us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.129:60593: BlockingRecv error: recv got EOF from 127.23.72.129:60593 (error 108)
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":103,"threads_started":1}
Aug 08 18:23:16 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.130@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:16 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.130@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:16.416934 1411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:16.411295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.130:48953)
0808 18:23:16.411580 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:16.411603 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:16.411709 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:16.411942 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:16.411944 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:16.412157 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:16.412263 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:16.412270 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:16.413033 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:16.413036 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:16.413657 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:16.413665 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:16.414088 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:16.414091 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:16.414161 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:16.414663 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:16.414677 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:16.416709 (+ 2032us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":96,"threads_started":1}
W20260808 18:23:16.416998 1410 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:16.411344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:48953 (local address 127.23.72.190:41351)
0808 18:23:16.411541 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:16.411545 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:16.411698 (+ 153us) server_negotiation.cc:407] Connection header received
0808 18:23:16.411778 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:16.411780 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:16.411829 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:16.411904 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:16.412373 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:16.412928 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:16.413766 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:16.413981 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:16.414055 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:16.416877 (+ 2822us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.130:48953: BlockingRecv error: recv got EOF from 127.23.72.130:48953 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 08 18:23:16 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.131@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:16 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.131@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:16.602715 1412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:16.597130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.131:48245)
0808 18:23:16.597444 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:16.597458 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:16.597556 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:16.597802 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:16.597804 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:16.597998 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:16.598090 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:16.598095 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:16.598862 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:16.598864 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:16.599407 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:16.599414 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:16.599839 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:16.599842 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:16.599904 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:16.600483 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:16.600494 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:16.602555 (+ 2061us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":131,"threads_started":1}
W20260808 18:23:16.602908 1410 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:16.597179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:48245 (local address 127.23.72.190:41351)
0808 18:23:16.597356 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:16.597361 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:16.597532 (+ 171us) server_negotiation.cc:407] Connection header received
0808 18:23:16.597645 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:16.597648 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:16.597696 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:16.597767 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:16.598227 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:16.598767 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:16.599533 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:16.599744 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:16.599824 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:16.602716 (+ 2892us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.131:48245: BlockingRecv error: recv got EOF from 127.23.72.131:48245 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 08 18:23:17 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.129@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:17 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.129@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:17.226984 1413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:17.220867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.129:50683)
0808 18:23:17.221205 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:17.221218 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:17.221317 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:17.221658 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:17.221660 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:17.221875 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:17.221975 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:17.221980 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:17.222751 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:17.222754 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:17.223363 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:17.223372 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:17.224003 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:17.224006 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:17.224074 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:17.224814 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:17.224827 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:17.226867 (+ 2040us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":92,"threads_started":1}
W20260808 18:23:17.227175 1414 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:17.220947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:50683 (local address 127.23.72.190:41351)
0808 18:23:17.221198 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:17.221201 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:17.221300 (+ 99us) server_negotiation.cc:407] Connection header received
0808 18:23:17.221471 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:17.221474 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:17.221526 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:17.221612 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:17.222087 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:17.222629 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:17.223572 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:17.223869 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:17.223971 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:17.227028 (+ 3057us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.129:50683: BlockingRecv error: recv got EOF from 127.23.72.129:50683 (error 108)
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":94,"threads_started":1}
Aug 08 18:23:17 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.130@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:17 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.130@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:17.423591 1415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:17.417844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.130:55335)
0808 18:23:17.418232 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:17.418247 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:17.418389 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:17.418668 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:17.418671 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:17.418895 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:17.418990 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:17.418995 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:17.419841 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:17.419846 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:17.420424 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:17.420433 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:17.420906 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:17.420910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:17.421000 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:17.421509 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:17.421520 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:17.423398 (+ 1878us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":88,"threads_started":1}
W20260808 18:23:17.423671 1414 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:17.417993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:55335 (local address 127.23.72.190:41351)
0808 18:23:17.418183 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:17.418188 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:17.418334 (+ 146us) server_negotiation.cc:407] Connection header received
0808 18:23:17.418500 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:17.418504 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:17.418557 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:17.418646 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:17.419115 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:17.419731 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:17.420557 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:17.420786 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:17.420892 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:17.423557 (+ 2665us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.130:55335: BlockingRecv error: recv got EOF from 127.23.72.130:55335 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 08 18:23:17 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.131@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:17 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.131@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:17.617483 1416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:17.603763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.131:36327)
0808 18:23:17.604148 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:17.604164 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:17.604293 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:17.604698 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:17.604704 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:17.605047 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:17.605173 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:17.605181 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:17.606095 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:17.606098 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:17.606655 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:17.606663 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:17.607100 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:17.607103 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:17.607168 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:17.614870 (+ 7702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:17.614883 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:17.617306 (+ 2423us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":116,"threads_started":1}
W20260808 18:23:17.617614 1414 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:17.603837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:36327 (local address 127.23.72.190:41351)
0808 18:23:17.604019 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:17.604024 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:17.604291 (+ 267us) server_negotiation.cc:407] Connection header received
0808 18:23:17.604493 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:17.604500 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:17.604570 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:17.604650 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:17.605307 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:17.605993 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:17.606760 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:17.606988 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:17.607068 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:17.617484 (+ 10416us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.131:36327: BlockingRecv error: recv got EOF from 127.23.72.131:36327 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 08 18:23:18 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.129@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:18 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.129@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:18.233861 1417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:18.227791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.129:50435)
0808 18:23:18.228240 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:18.228253 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:18.228342 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:18.228801 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:18.228805 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:18.229036 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:18.229139 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:18.229145 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:18.230047 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:18.230050 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:18.230617 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:18.230624 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:18.231059 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:18.231062 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:18.231127 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:18.231790 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:18.231803 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:18.233742 (+ 1939us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":124,"threads_started":1}
W20260808 18:23:18.234031 1418 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:18.227999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:50435 (local address 127.23.72.190:41351)
0808 18:23:18.228439 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:18.228443 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:18.228457 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:23:18.228500 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:18.228502 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:18.228544 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:18.228659 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:18.229281 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:18.229936 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:18.230730 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:18.230956 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:18.231036 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:18.233891 (+ 2855us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.129:50435: BlockingRecv error: recv got EOF from 127.23.72.129:50435 (error 108)
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":84,"threads_started":1}
Aug 08 18:23:18 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.130@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:18 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.130@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:18.430792 1421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:18.424449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.130:53043)
0808 18:23:18.424955 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:18.424972 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:18.425108 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:18.425389 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:18.425393 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:18.425637 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:18.425786 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:18.425796 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:18.426847 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:18.426852 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:18.427483 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:18.427493 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:18.428002 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:18.428005 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:18.428094 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:18.428625 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:18.428637 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:18.430667 (+ 2030us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":113,"threads_started":1}
W20260808 18:23:18.430998 1418 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:18.424629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:53043 (local address 127.23.72.190:41351)
0808 18:23:18.424790 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:18.424793 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:18.425072 (+ 279us) server_negotiation.cc:407] Connection header received
0808 18:23:18.425203 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:18.425206 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:18.425266 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:18.425360 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:18.425888 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:18.426737 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:18.427615 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:18.427879 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:18.427995 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:18.430857 (+ 2862us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.130:53043: BlockingRecv error: recv got EOF from 127.23.72.130:53043 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 08 18:23:18 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.131@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:18 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.131@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:18.624174 1422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:18.618325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.131:57659)
0808 18:23:18.618640 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:18.618654 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:18.618764 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:18.619024 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:18.619026 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:18.619213 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:18.619314 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:18.619319 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:18.620138 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:18.620140 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:18.620696 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:18.620704 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:18.621145 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:18.621147 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:18.621209 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:18.621715 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:18.621726 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:18.624015 (+ 2289us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":135,"threads_started":1}
W20260808 18:23:18.624279 1418 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:18.618456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:57659 (local address 127.23.72.190:41351)
0808 18:23:18.618604 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:18.618608 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:18.618731 (+ 123us) server_negotiation.cc:407] Connection header received
0808 18:23:18.618834 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:18.618836 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:18.618898 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:18.618973 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:18.619446 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:18.620037 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:18.620805 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:18.621041 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:18.621123 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:18.624152 (+ 3029us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.131:57659: BlockingRecv error: recv got EOF from 127.23.72.131:57659 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 08 18:23:19 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.129@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:19 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.129@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:19.240810 1423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:19.234759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.129:42277)
0808 18:23:19.235060 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:19.235074 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:19.235164 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:19.235448 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:19.235450 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:19.235655 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:19.235782 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:19.235789 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:19.236818 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:19.236821 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:19.237506 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:19.237516 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:19.237933 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:19.237935 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:19.237995 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:19.238585 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:19.238597 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:19.240686 (+ 2089us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":116,"threads_started":1}
W20260808 18:23:19.240984 1424 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:19.234843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:42277 (local address 127.23.72.190:41351)
0808 18:23:19.235124 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:19.235127 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:19.235142 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:23:19.235255 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:19.235257 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:19.235303 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:19.235407 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:19.235933 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:19.236712 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:19.237616 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:19.237834 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:19.237921 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:19.240820 (+ 2899us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.129:42277: BlockingRecv error: recv got EOF from 127.23.72.129:42277 (error 108)
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":118,"threads_started":1}
Aug 08 18:23:19 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.130@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:19 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.130@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:19.437870 1425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:19.431893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.130:56367)
0808 18:23:19.432248 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:19.432260 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:19.432417 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:19.432705 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:19.432708 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:19.432935 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:19.433030 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:19.433036 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:19.433860 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:19.433863 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:19.434420 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:19.434429 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:19.434917 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:19.434920 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:19.435008 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:19.435550 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:19.435562 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:19.437659 (+ 2097us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":86,"threads_started":1}
W20260808 18:23:19.437886 1424 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:19.431987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:56367 (local address 127.23.72.190:41351)
0808 18:23:19.432135 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:19.432140 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:19.432351 (+ 211us) server_negotiation.cc:407] Connection header received
0808 18:23:19.432525 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:19.432529 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:19.432586 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:19.432675 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:19.433154 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:19.433691 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:19.434551 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:19.434796 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:19.434905 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:19.437789 (+ 2884us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.130:56367: BlockingRecv error: recv got EOF from 127.23.72.130:56367 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 08 18:23:19 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.131@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:19 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.131@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:19.630978 1426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:19.625098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.131:59607)
0808 18:23:19.625417 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:19.625433 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:19.625575 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:19.625851 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:19.625854 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:19.626088 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:19.626182 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:19.626188 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:19.627114 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:19.627119 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:19.627751 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:19.627760 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:19.628225 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:19.628229 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:19.628316 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:19.628820 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:19.628861 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:19.630840 (+ 1979us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":109,"threads_started":1}
W20260808 18:23:19.631143 1424 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:19.625176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:59607 (local address 127.23.72.190:41351)
0808 18:23:19.625392 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:19.625397 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:19.625513 (+ 116us) server_negotiation.cc:407] Connection header received
0808 18:23:19.625679 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:19.625682 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:19.625736 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:19.625821 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:19.626307 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:19.626996 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:19.627878 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:19.628112 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:19.628223 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:19.630986 (+ 2763us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.131:59607: BlockingRecv error: recv got EOF from 127.23.72.131:59607 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
Aug 08 18:23:20 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.129@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:20 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.129@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:20.247171 1428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:20.241672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.129:54285)
0808 18:23:20.241994 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:20.242007 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:20.242103 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:20.242317 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:20.242318 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:20.242512 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:20.242606 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:20.242611 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:20.243487 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:20.243493 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:20.244060 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:20.244068 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:20.244572 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:20.244575 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:20.244636 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:20.245212 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:20.245224 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:20.247059 (+ 1835us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":143,"threads_started":1}
W20260808 18:23:20.247352 1427 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:20.241692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:54285 (local address 127.23.72.190:41351)
0808 18:23:20.242007 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:20.242010 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:20.242061 (+ 51us) server_negotiation.cc:407] Connection header received
0808 18:23:20.242183 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:20.242185 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:20.242232 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:20.242309 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:20.242708 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:20.243330 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:20.244224 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:20.244468 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:20.244556 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:20.247211 (+ 2655us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.129:54285: BlockingRecv error: recv got EOF from 127.23.72.129:54285 (error 108)
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":69,"threads_started":1}
Aug 08 18:23:20 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.130@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:20 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.130@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:20.444144 1429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:20.438720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.130:42307)
0808 18:23:20.439018 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:20.439043 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:20.439133 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:20.439344 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:20.439345 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:20.439550 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:20.439665 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:20.439670 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:20.440386 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:20.440389 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:20.441070 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:20.441080 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:20.441507 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:20.441509 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:20.441570 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:20.442051 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:20.442062 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:20.444004 (+ 1942us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":81,"threads_started":1}
W20260808 18:23:20.444257 1427 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:20.438780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:42307 (local address 127.23.72.190:41351)
0808 18:23:20.438922 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:20.438926 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:20.439092 (+ 166us) server_negotiation.cc:407] Connection header received
0808 18:23:20.439198 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:20.439200 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:20.439239 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:20.439309 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:20.439766 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:20.440289 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:20.441170 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:20.441421 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:20.441500 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:20.444142 (+ 2642us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.130:42307: BlockingRecv error: recv got EOF from 127.23.72.130:42307 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 08 18:23:20 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.131@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:20 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.131@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:20.637954 1430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:20.632061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.131:53117)
0808 18:23:20.632396 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:20.632425 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:20.632544 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:20.632898 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:20.632900 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:20.633108 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:20.633209 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:20.633215 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:20.633953 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:20.633956 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:20.634582 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:20.634590 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:20.635044 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:20.635047 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:20.635125 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:20.635793 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:20.635816 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:20.637832 (+ 2016us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":80,"threads_started":1}
W20260808 18:23:20.638140 1427 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:20.632261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:53117 (local address 127.23.72.190:41351)
0808 18:23:20.632453 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:20.632460 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:20.632481 (+ 21us) server_negotiation.cc:407] Connection header received
0808 18:23:20.632689 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:20.632694 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:20.632757 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:20.632837 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:20.633311 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:20.633845 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:20.634684 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:20.634932 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:20.635010 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:20.637977 (+ 2967us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.131:53117: BlockingRecv error: recv got EOF from 127.23.72.131:53117 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 08 18:23:21 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.129@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:21 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.129@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:21.254311 1431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:21.248210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.129:54147)
0808 18:23:21.248505 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:21.248518 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:21.248616 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:21.248866 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:21.248868 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:21.249064 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:21.249164 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:21.249169 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:21.249985 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:21.249988 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:21.250558 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:21.250566 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:21.251074 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:21.251077 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:21.251146 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:21.251849 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:21.251861 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:21.254128 (+ 2267us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":114,"threads_started":1}
W20260808 18:23:21.254391 1432 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:21.248255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:54147 (local address 127.23.72.190:41351)
0808 18:23:21.248514 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:21.248517 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:21.248585 (+ 68us) server_negotiation.cc:407] Connection header received
0808 18:23:21.248697 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:21.248699 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:21.248744 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:21.248822 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:21.249324 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:21.249869 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:21.250674 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:21.250929 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:21.251016 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:21.254262 (+ 3246us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.129:54147: BlockingRecv error: recv got EOF from 127.23.72.129:54147 (error 108)
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":98,"threads_started":1}
Aug 08 18:23:21 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.130@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:21 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.130@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:21.451150 1433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:21.445033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.130:54305)
0808 18:23:21.445393 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:21.445412 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:21.445553 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:21.445828 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:21.445831 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:21.446051 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:21.446144 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:21.446150 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:21.446919 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:21.446924 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:21.447577 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:21.447587 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:21.448102 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:21.448106 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:21.448193 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:21.448809 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:21.448827 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:21.451036 (+ 2209us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":86,"threads_started":1}
W20260808 18:23:21.451498 1432 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:21.445167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:54305 (local address 127.23.72.190:41351)
0808 18:23:21.445352 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:21.445357 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:21.445491 (+ 134us) server_negotiation.cc:407] Connection header received
0808 18:23:21.445656 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:21.445659 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:21.445708 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:21.445813 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:21.446265 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:21.446797 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:21.447698 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:21.447981 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:21.448098 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:21.451238 (+ 3140us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.130:54305: BlockingRecv error: recv got EOF from 127.23.72.130:54305 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 08 18:23:21 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.131@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:21 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.131@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:21.644510 1434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:21.638737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.131:46035)
0808 18:23:21.639030 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:21.639044 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:21.639145 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:21.639389 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:21.639392 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:21.639598 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:21.639698 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:21.639704 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:21.640575 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:21.640578 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:21.641192 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:21.641201 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:21.641665 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:21.641668 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:21.641727 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:21.642322 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:21.642334 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:21.644340 (+ 2006us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":107,"threads_started":1}
W20260808 18:23:21.644584 1432 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:21.638816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:46035 (local address 127.23.72.190:41351)
0808 18:23:21.638979 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:21.638983 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:21.639117 (+ 134us) server_negotiation.cc:407] Connection header received
0808 18:23:21.639224 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:21.639226 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:21.639276 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:21.639350 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:21.639839 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:21.640467 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:21.641299 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:21.641565 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:21.641648 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:21.644469 (+ 2821us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.131:46035: BlockingRecv error: recv got EOF from 127.23.72.131:46035 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 08 18:23:22 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.129@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:22 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.129@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:22.261133 1436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:22.255152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.129:36555)
0808 18:23:22.255469 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:22.255482 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:22.255578 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:22.255843 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:22.255845 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:22.256033 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:22.256140 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:22.256146 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:22.256889 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:22.256892 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:22.257467 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:22.257501 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:22.258072 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:22.258076 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:22.258174 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:22.258769 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:22.258784 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:22.260954 (+ 2170us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":103,"threads_started":1}
W20260808 18:23:22.261206 1437 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:22.255222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:36555 (local address 127.23.72.190:41351)
0808 18:23:22.255549 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:22.255552 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:22.255567 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:23:22.255663 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:22.255665 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:22.255710 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:22.255798 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:22.256237 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:22.256783 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:22.257614 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:22.257945 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:22.258062 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:22.261080 (+ 3018us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.129:36555: BlockingRecv error: recv got EOF from 127.23.72.129:36555 (error 108)
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":111,"threads_started":1}
Aug 08 18:23:22 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.130@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:22 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.130@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:22.457767 1438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:22.451915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.130:48261)
0808 18:23:22.452234 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:22.452252 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:22.452401 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:22.452687 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:22.452690 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:22.452921 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:22.453016 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:22.453021 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:22.453790 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:22.453797 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:22.454525 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:22.454536 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:22.454981 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:22.454985 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:22.455067 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:22.455740 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:22.455751 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:22.457649 (+ 1898us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":85,"threads_started":1}
W20260808 18:23:22.457928 1437 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:22.452051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:48261 (local address 127.23.72.190:41351)
0808 18:23:22.452231 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:22.452235 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:22.452353 (+ 118us) server_negotiation.cc:407] Connection header received
0808 18:23:22.452511 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:22.452514 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:22.452566 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:22.452659 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:22.453139 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:22.453675 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:22.454629 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:22.454866 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:22.454978 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:22.457790 (+ 2812us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.130:48261: BlockingRecv error: recv got EOF from 127.23.72.130:48261 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 08 18:23:22 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.131@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:22 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.131@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:22.654270 1440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:22.645313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.131:38699)
0808 18:23:22.645693 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:22.645707 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:22.645855 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:22.646141 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:22.646143 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:22.646364 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:22.646457 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:22.646463 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:22.647254 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:22.647261 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:22.647932 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:22.647941 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:22.648425 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:22.648428 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:22.648514 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:22.652177 (+ 3663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:22.652189 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:22.654152 (+ 1963us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":87,"threads_started":1}
W20260808 18:23:22.654435 1437 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:22.645460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:38699 (local address 127.23.72.190:41351)
0808 18:23:22.645643 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:22.645648 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:22.645786 (+ 138us) server_negotiation.cc:407] Connection header received
0808 18:23:22.645959 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:22.645963 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:22.646016 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:22.646114 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:22.646581 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:22.647139 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:22.648058 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:22.648302 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:22.648420 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:22.654292 (+ 5872us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.131:38699: BlockingRecv error: recv got EOF from 127.23.72.131:38699 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 08 18:23:23 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.129@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:23 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.129@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:23.267972 1441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:23.261988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.129:48949)
0808 18:23:23.262325 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:23.262338 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:23.262435 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:23.262716 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:23.262718 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:23.262926 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:23.263030 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:23.263035 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:23.264012 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:23.264018 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:23.264716 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:23.264729 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:23.265342 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:23.265345 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:23.265409 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:23.265907 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:23.265919 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:23.267832 (+ 1913us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":103,"threads_started":1}
W20260808 18:23:23.268101 1442 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:23.262119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:48949 (local address 127.23.72.190:41351)
0808 18:23:23.262383 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:23.262386 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:23.262401 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:23:23.262545 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:23.262547 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:23.262594 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:23.262673 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:23.263146 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:23.263867 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:23.264866 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:23.265226 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:23.265332 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:23.267971 (+ 2639us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.129:48949: BlockingRecv error: recv got EOF from 127.23.72.129:48949 (error 108)
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":63,"threads_started":1}
Aug 08 18:23:23 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.130@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:23 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.130@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:23.464507 1443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:23.458621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.130:38017)
0808 18:23:23.458976 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:23.458994 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:23.459133 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:23.459416 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:23.459419 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:23.459663 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:23.459754 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:23.459760 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:23.460524 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:23.460530 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:23.461216 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:23.461225 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:23.461749 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:23.461752 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:23.461841 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:23.462437 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:23.462457 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:23.464390 (+ 1933us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":117,"threads_started":1}
W20260808 18:23:23.464695 1442 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:23.458844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:38017 (local address 127.23.72.190:41351)
0808 18:23:23.459006 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:23.459012 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:23.459083 (+ 71us) server_negotiation.cc:407] Connection header received
0808 18:23:23.459239 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:23.459242 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:23.459296 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:23.459396 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:23.459881 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:23.460413 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:23.461340 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:23.461622 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:23.461733 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:23.464530 (+ 2797us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.130:38017: BlockingRecv error: recv got EOF from 127.23.72.130:38017 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 08 18:23:23 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.131@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:23 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.131@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:23.661665 1444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:23.655264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.131:36833)
0808 18:23:23.655598 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:23.655619 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:23.655784 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:23.656110 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:23.656113 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:23.656373 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:23.656476 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:23.656483 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:23.657327 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:23.657332 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:23.657980 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:23.657990 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:23.658600 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:23.658604 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:23.658701 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:23.659378 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:23.659395 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:23.661539 (+ 2144us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":86,"threads_started":1}
W20260808 18:23:23.661885 1442 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:23.655356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:36833 (local address 127.23.72.190:41351)
0808 18:23:23.655548 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:23.655553 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:23.655721 (+ 168us) server_negotiation.cc:407] Connection header received
0808 18:23:23.655931 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:23.655934 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:23.655986 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:23.656079 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:23.656610 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:23.657198 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:23.658119 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:23.658472 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:23.658596 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:23.661692 (+ 3096us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.131:36833: BlockingRecv error: recv got EOF from 127.23.72.131:36833 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 08 18:23:24 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.129@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:24 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.129@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:24.274675 1445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:24.268857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.129:33701)
0808 18:23:24.269154 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:24.269167 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:24.269257 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:24.269590 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:24.269592 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:24.269799 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:24.269913 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:24.269918 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:24.270765 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:24.270769 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:24.271656 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:24.271667 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:24.272089 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:24.272091 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:24.272152 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:24.272722 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:24.272734 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:24.274561 (+ 1827us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":117,"threads_started":1}
W20260808 18:23:24.274827 1446 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:24.269051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:33701 (local address 127.23.72.190:41351)
0808 18:23:24.269360 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:24.269364 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:24.269376 (+ 12us) server_negotiation.cc:407] Connection header received
0808 18:23:24.269412 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:24.269414 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:24.269455 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:24.269537 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:24.270024 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:24.270606 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:24.271765 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:24.271991 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:24.272069 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:24.274692 (+ 2623us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.129:33701: BlockingRecv error: recv got EOF from 127.23.72.129:33701 (error 108)
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":85,"threads_started":1}
Aug 08 18:23:24 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.130@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:24 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.130@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:24.471287 1447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:24.465412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.130:49743)
0808 18:23:24.465690 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:24.465703 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:24.465808 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:24.466051 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:24.466053 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:24.466241 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:24.466337 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:24.466342 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:24.467074 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:24.467076 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:24.467726 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:24.467737 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:24.468417 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:24.468421 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:24.468524 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:24.469162 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:24.469177 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:24.471160 (+ 1983us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":103,"threads_started":1}
W20260808 18:23:24.471458 1446 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:24.465522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:49743 (local address 127.23.72.190:41351)
0808 18:23:24.465671 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:24.465674 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:24.465778 (+ 104us) server_negotiation.cc:407] Connection header received
0808 18:23:24.465893 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:24.465895 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:24.465940 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:24.466010 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:24.466447 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:24.466978 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:24.467847 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:24.468138 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:24.468247 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:24.471294 (+ 3047us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.130:49743: BlockingRecv error: recv got EOF from 127.23.72.130:49743 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 08 18:23:24 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.131@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:24 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.131@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:24.668617 1448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:24.662696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.131:59433)
0808 18:23:24.663079 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:24.663091 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:24.663219 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:24.663564 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:24.663567 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:24.663833 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:24.663993 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:24.663999 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:24.664863 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:24.664867 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:24.665449 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:24.665457 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:24.665928 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:24.665931 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:24.666016 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:24.666526 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:24.666537 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:24.668491 (+ 1954us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":105,"threads_started":1}
W20260808 18:23:24.668871 1446 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:24.662777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:59433 (local address 127.23.72.190:41351)
0808 18:23:24.662939 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:24.662946 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:24.663304 (+ 358us) server_negotiation.cc:407] Connection header received
0808 18:23:24.663348 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:24.663351 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:24.663417 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:24.663542 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:24.664110 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:24.664736 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:24.665569 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:24.665811 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:24.665927 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:24.668632 (+ 2705us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.131:59433: BlockingRecv error: recv got EOF from 127.23.72.131:59433 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 08 18:23:25 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.129@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:25 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.129@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:25.281519 1449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:25.275691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.129:35483)
0808 18:23:25.276006 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:25.276020 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:25.276106 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:25.276329 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:25.276331 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:25.276534 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:25.276625 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:25.276630 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:25.277378 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:25.277380 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:25.277929 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:25.277936 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:25.278381 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:25.278384 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:25.278448 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:25.278948 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:25.278961 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:25.281400 (+ 2439us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":87,"threads_started":1}
W20260808 18:23:25.281682 1450 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:25.275815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:35483 (local address 127.23.72.190:41351)
0808 18:23:25.276094 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:25.276097 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:25.276111 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:23:25.276175 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:25.276177 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:25.276218 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:25.276293 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:25.276732 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:25.277271 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:25.278034 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:25.278266 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:25.278363 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:25.281541 (+ 3178us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.129:35483: BlockingRecv error: recv got EOF from 127.23.72.129:35483 (error 108)
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":72,"threads_started":1}
Aug 08 18:23:25 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.130@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:23:25 dist-test-slave-5k54 krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.72.130@KRBTEST.COM for oryx/127.23.72.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:23:25.477752 1451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:25.472239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41351 (local address 127.23.72.130:50109)
0808 18:23:25.472525 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:25.472538 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:25.472642 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:25.472868 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:25.472871 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:25.473061 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:25.473154 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:25.473159 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:25.473907 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:25.473909 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:25.474475 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:25.474482 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:25.474979 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:25.474985 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:25.475107 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:25.475641 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:25.475653 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:25.477555 (+ 1902us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.72.190:41351: Server oryx/127.23.72.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":111,"threads_started":1}
W20260808 18:23:25.477840 1450 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:25.472368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:50109 (local address 127.23.72.190:41351)
0808 18:23:25.472511 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:25.472514 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:25.472615 (+ 101us) server_negotiation.cc:407] Connection header received
0808 18:23:25.472717 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:25.472719 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:25.472760 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:25.472828 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:25.473253 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:25.473806 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:25.474570 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:25.474776 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:25.474920 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:25.477709 (+ 2789us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.72.130:50109: BlockingRecv error: recv got EOF from 127.23.72.130:50109 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 18:23:25.546435 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 933
I20260808 18:23:25.553848 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 1102
I20260808 18:23:25.560863 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 1241
I20260808 18:23:25.567700 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 855
2026-08-08T18:23:25Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18038 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-08T18:23:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:23:25Z Disabled control of system clock
I20260808 18:23:25.601758 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:33057
--webserver_interface=127.23.72.190
--webserver_port=0
--builtin_ntp_servers=127.23.72.148:38307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:33057
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 18:23:25.716658 1455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:23:25.716928 1455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:23:25.716960 1455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:23:25.720963 1455 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:23:25.721016 1455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:23:25.721032 1455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:23:25.721048 1455 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:23:25.721063 1455 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:23:25.725657 1455 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:38307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:33057
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:33057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.72.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1455
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:23:25.726655 1455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:23:25.728116 1455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:23:25.736613 1462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:23:25.736635 1461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:23:25.736639 1464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:23:25.736694 1455 server_base.cc:1034] running on GCE node
I20260808 18:23:25.737452 1455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:23:25.738583 1455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:23:25.739778 1455 hybrid_clock.cc:648] HybridClock initialized: now 1786213405739761 us; error 32 us; skew 500 ppm
I20260808 18:23:25.741639 1455 webserver.cc:460] Webserver started at http://127.23.72.190:35717/ using document root <none> and password file <none>
I20260808 18:23:25.742241 1455 fs_manager.cc:359] Metadata directory not provided
I20260808 18:23:25.742336 1455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:23:25.742568 1455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:23:25.744655 1455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "5294d79633804887820e1acba1889bef"
format_stamp: "Formatted at 2026-08-08 18:23:25 on dist-test-slave-5k54"
server_key: "b43d210b3b44ecd1d6f720a0f3454b95"
server_key_iv: "033e5524355ce1e9d6d6ab4d9edabfcc"
server_key_version: "encryptionkey@0"
I20260808 18:23:25.745250 1455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "5294d79633804887820e1acba1889bef"
format_stamp: "Formatted at 2026-08-08 18:23:25 on dist-test-slave-5k54"
server_key: "b43d210b3b44ecd1d6f720a0f3454b95"
server_key_iv: "033e5524355ce1e9d6d6ab4d9edabfcc"
server_key_version: "encryptionkey@0"
I20260808 18:23:25.748914 1455 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:23:25.751309 1470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:25.752575 1455 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:23:25.752713 1455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "5294d79633804887820e1acba1889bef"
format_stamp: "Formatted at 2026-08-08 18:23:25 on dist-test-slave-5k54"
server_key: "b43d210b3b44ecd1d6f720a0f3454b95"
server_key_iv: "033e5524355ce1e9d6d6ab4d9edabfcc"
server_key_version: "encryptionkey@0"
I20260808 18:23:25.752826 1455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:23:25.769421 1455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:23:25.770088 1455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:23:25.770282 1455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:23:25.777253 1455 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.190:33057
I20260808 18:23:25.777266 1522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.190:33057 every 8 connection(s)
I20260808 18:23:25.778203 1455 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:23:25.781389 1523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:23:25.786856 1523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5294d79633804887820e1acba1889bef: Bootstrap starting.
I20260808 18:23:25.788261 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 1455
I20260808 18:23:25.788396 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:23:25.788743 23842 external_mini_cluster.cc:1428] Setting key 9e170b21116ec6fbfcdd0a8ad96f61bf
I20260808 18:23:25.789695 1523 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5294d79633804887820e1acba1889bef: Neither blocks nor log segments found. Creating new log.
I20260808 18:23:25.790580 1523 log.cc:824] T 00000000000000000000000000000000 P 5294d79633804887820e1acba1889bef: Log is configured to *not* fsync() on all Append() calls
I20260808 18:23:25.792832 1523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5294d79633804887820e1acba1889bef: No bootstrap required, opened a new log
I20260808 18:23:25.797137 1523 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5294d79633804887820e1acba1889bef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5294d79633804887820e1acba1889bef" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 33057 } }
I20260808 18:23:25.797425 1523 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5294d79633804887820e1acba1889bef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:23:25.797489 1523 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5294d79633804887820e1acba1889bef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5294d79633804887820e1acba1889bef, State: Initialized, Role: FOLLOWER
I20260808 18:23:25.797991 1523 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5294d79633804887820e1acba1889bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5294d79633804887820e1acba1889bef" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 33057 } }
I20260808 18:23:25.798149 1523 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5294d79633804887820e1acba1889bef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:23:25.798223 1523 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5294d79633804887820e1acba1889bef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:23:25.798313 1523 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5294d79633804887820e1acba1889bef [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:23:25.799196 1526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:25.789962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38308 (local address 127.23.72.190:33057)
0808 18:23:25.790690 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:25.790712 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:25.790754 (+ 42us) server_negotiation.cc:407] Connection header received
0808 18:23:25.791592 (+ 838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:25.791613 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:25.792028 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:25.792386 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:25.792822 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:25.794323 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:25.795235 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:25.796035 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:25.796233 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:25.797019 (+ 786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:23:25.797026 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:23:25.797042 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:23:25.797544 (+ 502us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:23:25.797616 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:23:25.797641 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:23:25.797738 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:25.797921 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 18:23:25.798195 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":123,"threads_started":1}
I20260808 18:23:25.799461 1523 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5294d79633804887820e1acba1889bef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5294d79633804887820e1acba1889bef" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 33057 } }
I20260808 18:23:25.799798 1523 leader_election.cc:302] T 00000000000000000000000000000000 P 5294d79633804887820e1acba1889bef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5294d79633804887820e1acba1889bef; no voters:
I20260808 18:23:25.800128 1523 leader_election.cc:288] T 00000000000000000000000000000000 P 5294d79633804887820e1acba1889bef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:23:25.800222 1528 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5294d79633804887820e1acba1889bef [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:23:25.800397 1528 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5294d79633804887820e1acba1889bef [term 1 LEADER]: Becoming Leader. State: Replica: 5294d79633804887820e1acba1889bef, State: Running, Role: LEADER
I20260808 18:23:25.800670 1528 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5294d79633804887820e1acba1889bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5294d79633804887820e1acba1889bef" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 33057 } }
I20260808 18:23:25.802461 1523 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5294d79633804887820e1acba1889bef [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:23:25.802635 1530 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5294d79633804887820e1acba1889bef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5294d79633804887820e1acba1889bef. Latest consensus state: current_term: 1 leader_uuid: "5294d79633804887820e1acba1889bef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5294d79633804887820e1acba1889bef" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 33057 } } }
I20260808 18:23:25.802650 1529 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5294d79633804887820e1acba1889bef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5294d79633804887820e1acba1889bef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5294d79633804887820e1acba1889bef" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 33057 } } }
I20260808 18:23:25.802884 1530 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5294d79633804887820e1acba1889bef [sys.catalog]: This master's current role is: LEADER
I20260808 18:23:25.802973 1529 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5294d79633804887820e1acba1889bef [sys.catalog]: This master's current role is: LEADER
I20260808 18:23:25.803355 1532 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:23:25.806286 1532 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:23:25.811810 1532 catalog_manager.cc:1347] Generated new cluster ID: 2fcb59a00ccf4fae81caab9f7d7dd978
I20260808 18:23:25.811887 1532 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:23:25.824221 1532 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:23:25.825169 1532 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:23:25.830870 1532 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5294d79633804887820e1acba1889bef: Generated new TSK 0
I20260808 18:23:25.831665 1532 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:23:25.838752 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.129:0
--local_ip_for_outbound_sockets=127.23.72.129
--webserver_interface=127.23.72.129
--webserver_port=0
--tserver_master_addrs=127.23.72.190:33057
--builtin_ntp_servers=127.23.72.148:38307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 18:23:25.951726 1549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:23:25.951982 1549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:23:25.952018 1549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:23:25.956023 1549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:23:25.956070 1549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:23:25.956121 1549 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.129
I20260808 18:23:25.960916 1549 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:38307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.72.129
--webserver_port=0
--tserver_master_addrs=127.23.72.190:33057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1549
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.129
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:23:25.961926 1549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:23:25.963245 1549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:23:25.973459 1555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:23:25.973450 1554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:23:25.973471 1557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:23:25.973865 1549 server_base.cc:1034] running on GCE node
I20260808 18:23:25.974225 1549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:23:25.974881 1549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:23:25.976075 1549 hybrid_clock.cc:648] HybridClock initialized: now 1786213405976066 us; error 30 us; skew 500 ppm
I20260808 18:23:25.977763 1549 webserver.cc:460] Webserver started at http://127.23.72.129:40155/ using document root <none> and password file <none>
I20260808 18:23:25.978355 1549 fs_manager.cc:359] Metadata directory not provided
I20260808 18:23:25.978446 1549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:23:25.978675 1549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:23:25.980732 1549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/data/instance:
uuid: "ed011292badc417bbf768b06e10f1ea3"
format_stamp: "Formatted at 2026-08-08 18:23:25 on dist-test-slave-5k54"
server_key: "00b40f397d7e0614cbf53e3a51ef74b8"
server_key_iv: "deae4bf71241f9736cfa2ec1fc0d4b63"
server_key_version: "encryptionkey@0"
I20260808 18:23:25.981305 1549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance:
uuid: "ed011292badc417bbf768b06e10f1ea3"
format_stamp: "Formatted at 2026-08-08 18:23:25 on dist-test-slave-5k54"
server_key: "00b40f397d7e0614cbf53e3a51ef74b8"
server_key_iv: "deae4bf71241f9736cfa2ec1fc0d4b63"
server_key_version: "encryptionkey@0"
I20260808 18:23:25.984887 1549 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:23:25.987497 1564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:25.988660 1549 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:23:25.988790 1549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/wal
uuid: "ed011292badc417bbf768b06e10f1ea3"
format_stamp: "Formatted at 2026-08-08 18:23:25 on dist-test-slave-5k54"
server_key: "00b40f397d7e0614cbf53e3a51ef74b8"
server_key_iv: "deae4bf71241f9736cfa2ec1fc0d4b63"
server_key_version: "encryptionkey@0"
I20260808 18:23:25.988906 1549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:23:26.013111 1549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:23:26.013837 1549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:23:26.014053 1549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:23:26.014642 1549 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:23:26.015610 1549 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:23:26.015693 1549 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:26.015787 1549 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:23:26.015844 1549 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:26.025144 1549 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.129:44375
I20260808 18:23:26.025183 1686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.129:44375 every 8 connection(s)
I20260808 18:23:26.026032 1549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
I20260808 18:23:26.034421 1526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:26.027734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:41191 (local address 127.23.72.190:33057)
0808 18:23:26.027899 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:26.027903 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:26.028918 (+ 1015us) server_negotiation.cc:407] Connection header received
0808 18:23:26.030229 (+ 1311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:26.030232 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:26.030279 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:26.030352 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:26.031138 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:26.031669 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:26.032720 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:26.032984 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:26.033062 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:26.034009 (+ 947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:23:26.034011 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:23:26.034013 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:23:26.034025 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:23:26.034061 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:23:26.034069 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:23:26.034125 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:26.034260 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 18:23:26.034308 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 18:23:26.035223 1689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:26.028112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:33057 (local address 127.23.72.129:41191)
0808 18:23:26.028747 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:26.028788 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:26.030046 (+ 1258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:26.030487 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:26.030493 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:26.030568 (+ 75us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:26.031019 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:26.031031 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:26.031774 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:26.031776 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:26.032613 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:26.032622 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:26.033110 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:26.033113 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:26.033245 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:26.033773 (+ 528us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:23:26.033774 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:26.033810 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:23:26.033873 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:23:26.034140 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:23:26.034147 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:23:26.034151 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 18:23:26.034238 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 18:23:26.034440 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":123,"threads_started":1}
I20260808 18:23:26.035952 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 1549
I20260808 18:23:26.036051 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance
I20260808 18:23:26.036375 23842 external_mini_cluster.cc:1428] Setting key 2a9e251357542c3ee1df14107bc55e92
I20260808 18:23:26.036558 1687 heartbeater.cc:347] Connected to a master server at 127.23.72.190:33057
I20260808 18:23:26.036778 1687 heartbeater.cc:464] Registering TS with master...
I20260808 18:23:26.037405 1687 heartbeater.cc:511] Master 127.23.72.190:33057 requested a full tablet report, sending...
I20260808 18:23:26.038408 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.130:0
--local_ip_for_outbound_sockets=127.23.72.130
--webserver_interface=127.23.72.130
--webserver_port=0
--tserver_master_addrs=127.23.72.190:33057
--builtin_ntp_servers=127.23.72.148:38307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 18:23:26.038954 1487 ts_manager.cc:194] Registered new tserver with Master: ed011292badc417bbf768b06e10f1ea3 (127.23.72.129:44375)
I20260808 18:23:26.040269 1487 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.72.129:41191
W20260808 18:23:26.151234 1690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:23:26.151556 1690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:23:26.151597 1690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:23:26.155534 1690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:23:26.155577 1690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:23:26.155627 1690 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.130
I20260808 18:23:26.160290 1690 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:38307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.72.130
--webserver_port=0
--tserver_master_addrs=127.23.72.190:33057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1690
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.130
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:23:26.161360 1690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:23:26.162675 1690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:23:26.172479 1699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:23:26.172483 1702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:23:26.172483 1698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:23:26.172547 1690 server_base.cc:1034] running on GCE node
I20260808 18:23:26.173276 1690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:23:26.173903 1690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:23:26.175076 1690 hybrid_clock.cc:648] HybridClock initialized: now 1786213406175058 us; error 36 us; skew 500 ppm
I20260808 18:23:26.177136 1690 webserver.cc:460] Webserver started at http://127.23.72.130:40939/ using document root <none> and password file <none>
I20260808 18:23:26.177738 1690 fs_manager.cc:359] Metadata directory not provided
I20260808 18:23:26.177802 1690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:23:26.178046 1690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:23:26.180200 1690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/data/instance:
uuid: "f5163e46ef724e688e29365e52275d85"
format_stamp: "Formatted at 2026-08-08 18:23:26 on dist-test-slave-5k54"
server_key: "bf31b149881d28768756dcbaf663e1e2"
server_key_iv: "28bd7b1e4ab63bcf5d1fec64ff8ae7a4"
server_key_version: "encryptionkey@0"
I20260808 18:23:26.180773 1690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance:
uuid: "f5163e46ef724e688e29365e52275d85"
format_stamp: "Formatted at 2026-08-08 18:23:26 on dist-test-slave-5k54"
server_key: "bf31b149881d28768756dcbaf663e1e2"
server_key_iv: "28bd7b1e4ab63bcf5d1fec64ff8ae7a4"
server_key_version: "encryptionkey@0"
I20260808 18:23:26.184422 1690 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:23:26.187016 1708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:26.188372 1690 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:23:26.188545 1690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/wal
uuid: "f5163e46ef724e688e29365e52275d85"
format_stamp: "Formatted at 2026-08-08 18:23:26 on dist-test-slave-5k54"
server_key: "bf31b149881d28768756dcbaf663e1e2"
server_key_iv: "28bd7b1e4ab63bcf5d1fec64ff8ae7a4"
server_key_version: "encryptionkey@0"
I20260808 18:23:26.188663 1690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:23:26.212463 1690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:23:26.213199 1690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:23:26.213415 1690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:23:26.214025 1690 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:23:26.214991 1690 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:23:26.215062 1690 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:26.215133 1690 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:23:26.215176 1690 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:26.224038 1690 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.130:33777
I20260808 18:23:26.224066 1832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.130:33777 every 8 connection(s)
I20260808 18:23:26.224949 1690 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
I20260808 18:23:26.225966 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 1690
I20260808 18:23:26.226083 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance
I20260808 18:23:26.226445 23842 external_mini_cluster.cc:1428] Setting key 951b9b63a237025cad7cf690dc49cbc8
I20260808 18:23:26.228595 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.131:0
--local_ip_for_outbound_sockets=127.23.72.131
--webserver_interface=127.23.72.131
--webserver_port=0
--tserver_master_addrs=127.23.72.190:33057
--builtin_ntp_servers=127.23.72.148:38307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 18:23:26.235625 1526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:26.227010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:33835 (local address 127.23.72.190:33057)
0808 18:23:26.227168 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:26.227172 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:26.228417 (+ 1245us) server_negotiation.cc:407] Connection header received
0808 18:23:26.229918 (+ 1501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:26.229924 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:26.229987 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:26.230083 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:26.231006 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:26.231903 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:26.233109 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:26.233532 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:26.233668 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:26.234865 (+ 1197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:23:26.234872 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:23:26.234878 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:23:26.234897 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:23:26.234924 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:23:26.234938 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:23:26.235027 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:26.235187 (+ 160us) server_negotiation.cc:299] Negotiation successful
0808 18:23:26.235268 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 18:23:26.236346 1835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:26.227521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:33057 (local address 127.23.72.130:33835)
0808 18:23:26.228247 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:26.228300 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:26.229704 (+ 1404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:26.230241 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:26.230247 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:26.230329 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:26.230840 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:26.230876 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:26.232080 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:26.232082 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:26.232981 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:26.232990 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:26.233723 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:26.233727 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:26.233924 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:26.234587 (+ 663us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:23:26.234589 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:26.234630 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:23:26.234693 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:23:26.235034 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:23:26.235041 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:23:26.235045 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 18:23:26.235141 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 18:23:26.235373 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":143,"threads_started":1}
I20260808 18:23:26.237582 1833 heartbeater.cc:347] Connected to a master server at 127.23.72.190:33057
I20260808 18:23:26.237854 1833 heartbeater.cc:464] Registering TS with master...
I20260808 18:23:26.238549 1833 heartbeater.cc:511] Master 127.23.72.190:33057 requested a full tablet report, sending...
I20260808 18:23:26.239776 1487 ts_manager.cc:194] Registered new tserver with Master: f5163e46ef724e688e29365e52275d85 (127.23.72.130:33777)
I20260808 18:23:26.240391 1487 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.72.130:33835
W20260808 18:23:26.347366 1837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:23:26.347724 1837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:23:26.347796 1837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:23:26.352308 1837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:23:26.352365 1837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:23:26.352463 1837 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.131
I20260808 18:23:26.357721 1837 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:38307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.72.131
--webserver_port=0
--tserver_master_addrs=127.23.72.190:33057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.131
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:23:26.358853 1837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:23:26.360389 1837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:23:26.370085 1845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:23:26.370177 1837 server_base.cc:1034] running on GCE node
W20260808 18:23:26.370088 1842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:23:26.370081 1843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:23:26.370905 1837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:23:26.371650 1837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:23:26.372830 1837 hybrid_clock.cc:648] HybridClock initialized: now 1786213406372812 us; error 26 us; skew 500 ppm
I20260808 18:23:26.374717 1837 webserver.cc:460] Webserver started at http://127.23.72.131:44461/ using document root <none> and password file <none>
I20260808 18:23:26.375329 1837 fs_manager.cc:359] Metadata directory not provided
I20260808 18:23:26.375397 1837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:23:26.375701 1837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:23:26.377969 1837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/data/instance:
uuid: "c23c9fb1c8474bab99e5405a3a335b7c"
format_stamp: "Formatted at 2026-08-08 18:23:26 on dist-test-slave-5k54"
server_key: "204cc23b6349f3a319cc8b00ec175d41"
server_key_iv: "aab6fd0da70f256bbd2363091ad247b0"
server_key_version: "encryptionkey@0"
I20260808 18:23:26.378618 1837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance:
uuid: "c23c9fb1c8474bab99e5405a3a335b7c"
format_stamp: "Formatted at 2026-08-08 18:23:26 on dist-test-slave-5k54"
server_key: "204cc23b6349f3a319cc8b00ec175d41"
server_key_iv: "aab6fd0da70f256bbd2363091ad247b0"
server_key_version: "encryptionkey@0"
I20260808 18:23:26.382513 1837 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 18:23:26.385206 1854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:26.386559 1837 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:23:26.386686 1837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/wal
uuid: "c23c9fb1c8474bab99e5405a3a335b7c"
format_stamp: "Formatted at 2026-08-08 18:23:26 on dist-test-slave-5k54"
server_key: "204cc23b6349f3a319cc8b00ec175d41"
server_key_iv: "aab6fd0da70f256bbd2363091ad247b0"
server_key_version: "encryptionkey@0"
I20260808 18:23:26.386806 1837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:23:26.427575 1837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:23:26.428392 1837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:23:26.428620 1837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:23:26.429294 1837 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:23:26.430357 1837 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:23:26.430409 1837 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:26.430455 1837 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:23:26.430511 1837 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:26.440714 1837 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.131:42467
I20260808 18:23:26.440773 1976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.131:42467 every 8 connection(s)
I20260808 18:23:26.441713 1837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
I20260808 18:23:26.446506 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 1837
I20260808 18:23:26.446597 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance
I20260808 18:23:26.446900 23842 external_mini_cluster.cc:1428] Setting key 0a66e8114963d98933e6a12ac63d776b
I20260808 18:23:26.451206 1526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:26.443610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:36677 (local address 127.23.72.190:33057)
0808 18:23:26.443767 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:26.443772 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:26.444791 (+ 1019us) server_negotiation.cc:407] Connection header received
0808 18:23:26.446251 (+ 1460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:26.446255 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:26.446317 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:26.446435 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:26.447537 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:26.448195 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:26.449507 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:26.449730 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:26.449814 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:26.450810 (+ 996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:23:26.450813 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:23:26.450815 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:23:26.450826 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:23:26.450844 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:23:26.450851 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:23:26.450916 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:26.451050 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 18:23:26.451097 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 18:23:26.452131 1979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:26.443959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:33057 (local address 127.23.72.131:36677)
0808 18:23:26.444615 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:26.444657 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:26.446035 (+ 1378us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:26.446634 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:26.446648 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:26.446767 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:26.447374 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:26.447396 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:26.448332 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:26.448337 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:26.449399 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:26.449409 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:26.449893 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:26.449896 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:26.450035 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:26.450580 (+ 545us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:23:26.450582 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:26.450621 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:23:26.450681 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:23:26.450931 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:23:26.450938 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:23:26.450942 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 18:23:26.451035 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 18:23:26.451244 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":109,"threads_started":1}
I20260808 18:23:26.453368 1977 heartbeater.cc:347] Connected to a master server at 127.23.72.190:33057
I20260808 18:23:26.453603 1977 heartbeater.cc:464] Registering TS with master...
I20260808 18:23:26.454100 1977 heartbeater.cc:511] Master 127.23.72.190:33057 requested a full tablet report, sending...
I20260808 18:23:26.455180 1487 ts_manager.cc:194] Registered new tserver with Master: c23c9fb1c8474bab99e5405a3a335b7c (127.23.72.131:42467)
I20260808 18:23:26.455787 1487 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.72.131:36677
I20260808 18:23:26.460757 23842 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 18:23:26.463743 1526 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:26.462677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38320 (local address 127.23.72.190:33057)
0808 18:23:26.462819 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:26.462823 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:26.463038 (+ 215us) server_negotiation.cc:407] Connection header received
0808 18:23:26.463188 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:26.463191 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:26.463238 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:26.463319 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:26.463621 (+ 302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38320: BlockingRecv error: recv got EOF from 127.0.0.1:38320 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 18:23:26.464008 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 1549
I20260808 18:23:26.472287 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 1690
I20260808 18:23:26.478459 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 1837
I20260808 18:23:26.485960 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 1455
2026-08-08T18:23:26Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (918 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-08T18:23:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:23:26Z Disabled control of system clock
I20260808 18:23:26.519073 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:44055
--webserver_interface=127.23.72.190
--webserver_port=0
--builtin_ntp_servers=127.23.72.148:40127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:44055
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 18:23:26.631733 1988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:23:26.632004 1988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:23:26.632040 1988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:23:26.636111 1988 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:23:26.636165 1988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:23:26.636181 1988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:23:26.636198 1988 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:23:26.636214 1988 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:23:26.640908 1988 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:40127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:44055
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:44055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.72.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:23:26.641907 1988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:23:26.643232 1988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:23:26.651678 1996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:23:26.651684 1993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:23:26.651684 1994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:23:26.651800 1988 server_base.cc:1034] running on GCE node
I20260808 18:23:26.652431 1988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:23:26.653468 1988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:23:26.654635 1988 hybrid_clock.cc:648] HybridClock initialized: now 1786213406654617 us; error 30 us; skew 500 ppm
I20260808 18:23:26.656502 1988 webserver.cc:460] Webserver started at http://127.23.72.190:44929/ using document root <none> and password file <none>
I20260808 18:23:26.657085 1988 fs_manager.cc:359] Metadata directory not provided
I20260808 18:23:26.657147 1988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:23:26.657398 1988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:23:26.659550 1988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "b62a0fb1efb54e8f8e2d4be8afa62f6a"
format_stamp: "Formatted at 2026-08-08 18:23:26 on dist-test-slave-5k54"
server_key: "5e155e6f887079c68d798ba37f93ab67"
server_key_iv: "7506903c95b1969486ac0489945fae02"
server_key_version: "encryptionkey@0"
I20260808 18:23:26.660156 1988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "b62a0fb1efb54e8f8e2d4be8afa62f6a"
format_stamp: "Formatted at 2026-08-08 18:23:26 on dist-test-slave-5k54"
server_key: "5e155e6f887079c68d798ba37f93ab67"
server_key_iv: "7506903c95b1969486ac0489945fae02"
server_key_version: "encryptionkey@0"
I20260808 18:23:26.663880 1988 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:23:26.666272 2002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:26.667544 1988 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:23:26.667685 1988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "b62a0fb1efb54e8f8e2d4be8afa62f6a"
format_stamp: "Formatted at 2026-08-08 18:23:26 on dist-test-slave-5k54"
server_key: "5e155e6f887079c68d798ba37f93ab67"
server_key_iv: "7506903c95b1969486ac0489945fae02"
server_key_version: "encryptionkey@0"
I20260808 18:23:26.667804 1988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:23:26.672356 1988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:23:26.672972 1988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:23:26.673156 1988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:23:26.680490 1988 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.190:44055
I20260808 18:23:26.680513 2058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.190:44055 every 8 connection(s)
I20260808 18:23:26.681469 1988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:23:26.684993 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 1988
I20260808 18:23:26.685091 2059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:23:26.685122 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:23:26.685464 23842 external_mini_cluster.cc:1428] Setting key 743f7445a25a53eca753a18955b9814d
I20260808 18:23:26.691172 2062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:26.686968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58484 (local address 127.23.72.190:44055)
0808 18:23:26.687683 (+ 715us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:26.687699 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:26.687735 (+ 36us) server_negotiation.cc:407] Connection header received
0808 18:23:26.688422 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:26.688438 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:26.688809 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:26.689150 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:26.689969 (+ 819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:23:26.689974 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:23:26.689985 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:23:26.689996 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:23:26.690051 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:23:26.690066 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:23:26.690216 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:26.690285 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 18:23:26.690523 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":138,"threads_started":1}
I20260808 18:23:26.691840 2059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b62a0fb1efb54e8f8e2d4be8afa62f6a: Bootstrap starting.
I20260808 18:23:26.694329 2059 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b62a0fb1efb54e8f8e2d4be8afa62f6a: Neither blocks nor log segments found. Creating new log.
I20260808 18:23:26.695065 2059 log.cc:824] T 00000000000000000000000000000000 P b62a0fb1efb54e8f8e2d4be8afa62f6a: Log is configured to *not* fsync() on all Append() calls
I20260808 18:23:26.697016 2059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b62a0fb1efb54e8f8e2d4be8afa62f6a: No bootstrap required, opened a new log
I20260808 18:23:26.700816 2059 raft_consensus.cc:348] T 00000000000000000000000000000000 P b62a0fb1efb54e8f8e2d4be8afa62f6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b62a0fb1efb54e8f8e2d4be8afa62f6a" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 44055 } }
I20260808 18:23:26.701068 2059 raft_consensus.cc:374] T 00000000000000000000000000000000 P b62a0fb1efb54e8f8e2d4be8afa62f6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:23:26.701157 2059 raft_consensus.cc:729] T 00000000000000000000000000000000 P b62a0fb1efb54e8f8e2d4be8afa62f6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b62a0fb1efb54e8f8e2d4be8afa62f6a, State: Initialized, Role: FOLLOWER
I20260808 18:23:26.701619 2059 consensus_queue.cc:260] T 00000000000000000000000000000000 P b62a0fb1efb54e8f8e2d4be8afa62f6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b62a0fb1efb54e8f8e2d4be8afa62f6a" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 44055 } }
I20260808 18:23:26.701771 2059 raft_consensus.cc:388] T 00000000000000000000000000000000 P b62a0fb1efb54e8f8e2d4be8afa62f6a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:23:26.701858 2059 raft_consensus.cc:482] T 00000000000000000000000000000000 P b62a0fb1efb54e8f8e2d4be8afa62f6a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:23:26.701970 2059 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b62a0fb1efb54e8f8e2d4be8afa62f6a [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:23:26.702935 2059 raft_consensus.cc:504] T 00000000000000000000000000000000 P b62a0fb1efb54e8f8e2d4be8afa62f6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b62a0fb1efb54e8f8e2d4be8afa62f6a" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 44055 } }
I20260808 18:23:26.703274 2059 leader_election.cc:302] T 00000000000000000000000000000000 P b62a0fb1efb54e8f8e2d4be8afa62f6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b62a0fb1efb54e8f8e2d4be8afa62f6a; no voters:
I20260808 18:23:26.703612 2059 leader_election.cc:288] T 00000000000000000000000000000000 P b62a0fb1efb54e8f8e2d4be8afa62f6a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:23:26.703732 2064 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b62a0fb1efb54e8f8e2d4be8afa62f6a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:23:26.704006 2064 raft_consensus.cc:686] T 00000000000000000000000000000000 P b62a0fb1efb54e8f8e2d4be8afa62f6a [term 1 LEADER]: Becoming Leader. State: Replica: b62a0fb1efb54e8f8e2d4be8afa62f6a, State: Running, Role: LEADER
I20260808 18:23:26.704561 2064 consensus_queue.cc:237] T 00000000000000000000000000000000 P b62a0fb1efb54e8f8e2d4be8afa62f6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b62a0fb1efb54e8f8e2d4be8afa62f6a" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 44055 } }
I20260808 18:23:26.704939 2059 sys_catalog.cc:564] T 00000000000000000000000000000000 P b62a0fb1efb54e8f8e2d4be8afa62f6a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:23:26.706068 2065 sys_catalog.cc:455] T 00000000000000000000000000000000 P b62a0fb1efb54e8f8e2d4be8afa62f6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b62a0fb1efb54e8f8e2d4be8afa62f6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b62a0fb1efb54e8f8e2d4be8afa62f6a" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 44055 } } }
I20260808 18:23:26.706188 2065 sys_catalog.cc:458] T 00000000000000000000000000000000 P b62a0fb1efb54e8f8e2d4be8afa62f6a [sys.catalog]: This master's current role is: LEADER
I20260808 18:23:26.706564 2067 sys_catalog.cc:455] T 00000000000000000000000000000000 P b62a0fb1efb54e8f8e2d4be8afa62f6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b62a0fb1efb54e8f8e2d4be8afa62f6a. Latest consensus state: current_term: 1 leader_uuid: "b62a0fb1efb54e8f8e2d4be8afa62f6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b62a0fb1efb54e8f8e2d4be8afa62f6a" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 44055 } } }
I20260808 18:23:26.706646 2067 sys_catalog.cc:458] T 00000000000000000000000000000000 P b62a0fb1efb54e8f8e2d4be8afa62f6a [sys.catalog]: This master's current role is: LEADER
I20260808 18:23:26.706672 2074 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:23:26.709578 2074 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:23:26.714591 2074 catalog_manager.cc:1347] Generated new cluster ID: 9b5cb818cd3a4e32a6c2af7655c8147c
I20260808 18:23:26.714654 2074 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:23:26.723701 2074 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:23:26.724035 2074 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:23:26.733294 2074 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b62a0fb1efb54e8f8e2d4be8afa62f6a: Generated new TSK 0
I20260808 18:23:26.733961 2074 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:23:26.736330 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.129:0
--local_ip_for_outbound_sockets=127.23.72.129
--webserver_interface=127.23.72.129
--webserver_port=0
--tserver_master_addrs=127.23.72.190:44055
--builtin_ntp_servers=127.23.72.148:40127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 18:23:26.866947 2085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:23:26.867204 2085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:23:26.867244 2085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:23:26.871541 2085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:23:26.871594 2085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:23:26.871676 2085 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.129
I20260808 18:23:26.878343 2085 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:40127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.72.129
--webserver_port=0
--tserver_master_addrs=127.23.72.190:44055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2085
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.129
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:23:26.879376 2085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:23:26.880785 2085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:23:26.890337 2095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:23:26.890337 2092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:23:26.890367 2093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:23:26.890545 2085 server_base.cc:1034] running on GCE node
I20260808 18:23:26.891021 2085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:23:26.891721 2085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:23:26.892877 2085 hybrid_clock.cc:648] HybridClock initialized: now 1786213406892848 us; error 41 us; skew 500 ppm
I20260808 18:23:26.894771 2085 webserver.cc:460] Webserver started at http://127.23.72.129:44723/ using document root <none> and password file <none>
I20260808 18:23:26.895350 2085 fs_manager.cc:359] Metadata directory not provided
I20260808 18:23:26.895412 2085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:23:26.895705 2085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:23:26.897760 2085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/data/instance:
uuid: "e7c6c4d3dcdc4bb3ab8c0d063940667d"
format_stamp: "Formatted at 2026-08-08 18:23:26 on dist-test-slave-5k54"
server_key: "6e8ae74d90cf08a2b0f0138bc013950d"
server_key_iv: "bea6127cc1b94878cd9bf3a8a8613c03"
server_key_version: "encryptionkey@0"
I20260808 18:23:26.898330 2085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance:
uuid: "e7c6c4d3dcdc4bb3ab8c0d063940667d"
format_stamp: "Formatted at 2026-08-08 18:23:26 on dist-test-slave-5k54"
server_key: "6e8ae74d90cf08a2b0f0138bc013950d"
server_key_iv: "bea6127cc1b94878cd9bf3a8a8613c03"
server_key_version: "encryptionkey@0"
I20260808 18:23:26.901990 2085 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260808 18:23:26.904561 2101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:26.905897 2085 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:23:26.906028 2085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/wal
uuid: "e7c6c4d3dcdc4bb3ab8c0d063940667d"
format_stamp: "Formatted at 2026-08-08 18:23:26 on dist-test-slave-5k54"
server_key: "6e8ae74d90cf08a2b0f0138bc013950d"
server_key_iv: "bea6127cc1b94878cd9bf3a8a8613c03"
server_key_version: "encryptionkey@0"
I20260808 18:23:26.906153 2085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:23:26.910905 2085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:23:26.911571 2085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:23:26.911746 2085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:23:26.912264 2085 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:23:26.913156 2085 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:23:26.913224 2085 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:26.913298 2085 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:23:26.913358 2085 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:26.922048 2085 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.129:38765
I20260808 18:23:26.922073 2224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.129:38765 every 8 connection(s)
I20260808 18:23:26.922935 2085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
I20260808 18:23:26.924562 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 2085
I20260808 18:23:26.924690 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance
I20260808 18:23:26.925055 23842 external_mini_cluster.cc:1428] Setting key 44a0cd67bae522889ada39a1ea39bf27
I20260808 18:23:26.927523 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.130:0
--local_ip_for_outbound_sockets=127.23.72.130
--webserver_interface=127.23.72.130
--webserver_port=0
--tserver_master_addrs=127.23.72.190:44055
--builtin_ntp_servers=127.23.72.148:40127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 18:23:26.929829 2062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:26.926485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:47569 (local address 127.23.72.190:44055)
0808 18:23:26.926622 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:26.926627 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:26.926641 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:23:26.928062 (+ 1421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:26.928067 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:26.928116 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:26.928213 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:26.929342 (+ 1129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:23:26.929345 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:23:26.929348 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:23:26.929362 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:23:26.929390 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:23:26.929397 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:23:26.929495 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:26.929646 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 18:23:26.929693 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 18:23:26.930697 2227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:26.925488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:44055 (local address 127.23.72.129:47569)
0808 18:23:26.926330 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:26.926405 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:26.927850 (+ 1445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:26.928381 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:26.928388 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:26.928477 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:26.929080 (+ 603us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:23:26.929081 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:26.929121 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:23:26.929188 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:23:26.929481 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:23:26.929490 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:23:26.929496 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 18:23:26.929695 (+ 199us) client_negotiation.cc:240] Negotiation successful
0808 18:23:26.929945 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":151,"threads_started":1}
I20260808 18:23:26.932101 2225 heartbeater.cc:347] Connected to a master server at 127.23.72.190:44055
I20260808 18:23:26.932370 2225 heartbeater.cc:464] Registering TS with master...
I20260808 18:23:26.932803 2225 heartbeater.cc:511] Master 127.23.72.190:44055 requested a full tablet report, sending...
I20260808 18:23:26.934653 2020 ts_manager.cc:194] Registered new tserver with Master: e7c6c4d3dcdc4bb3ab8c0d063940667d (127.23.72.129:38765)
W20260808 18:23:27.042270 2228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:23:27.042539 2228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:23:27.042577 2228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:23:27.046496 2228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:23:27.046546 2228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:23:27.046600 2228 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.130
I20260808 18:23:27.051167 2228 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:40127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.72.130
--webserver_port=0
--tserver_master_addrs=127.23.72.190:44055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2228
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.130
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:23:27.052258 2228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:23:27.053608 2228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:23:27.062956 2234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:23:27.062956 2236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:23:27.062975 2233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:23:27.063130 2228 server_base.cc:1034] running on GCE node
I20260808 18:23:27.063684 2228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:23:27.064304 2228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:23:27.065490 2228 hybrid_clock.cc:648] HybridClock initialized: now 1786213407065479 us; error 27 us; skew 500 ppm
I20260808 18:23:27.067359 2228 webserver.cc:460] Webserver started at http://127.23.72.130:35997/ using document root <none> and password file <none>
I20260808 18:23:27.067970 2228 fs_manager.cc:359] Metadata directory not provided
I20260808 18:23:27.068065 2228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:23:27.068300 2228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:23:27.070323 2228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/data/instance:
uuid: "06f7109e693c44bfaac7349d5e1fa5d1"
format_stamp: "Formatted at 2026-08-08 18:23:27 on dist-test-slave-5k54"
server_key: "2f78f5727521bb4c5d907db59a49343e"
server_key_iv: "620887d066b41a3221955da56f7d7484"
server_key_version: "encryptionkey@0"
I20260808 18:23:27.070904 2228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance:
uuid: "06f7109e693c44bfaac7349d5e1fa5d1"
format_stamp: "Formatted at 2026-08-08 18:23:27 on dist-test-slave-5k54"
server_key: "2f78f5727521bb4c5d907db59a49343e"
server_key_iv: "620887d066b41a3221955da56f7d7484"
server_key_version: "encryptionkey@0"
I20260808 18:23:27.074556 2228 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 18:23:27.077069 2242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:27.078387 2228 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:23:27.078505 2228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/wal
uuid: "06f7109e693c44bfaac7349d5e1fa5d1"
format_stamp: "Formatted at 2026-08-08 18:23:27 on dist-test-slave-5k54"
server_key: "2f78f5727521bb4c5d907db59a49343e"
server_key_iv: "620887d066b41a3221955da56f7d7484"
server_key_version: "encryptionkey@0"
I20260808 18:23:27.078625 2228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:23:27.083112 2228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:23:27.083783 2228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:23:27.083968 2228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:23:27.084522 2228 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:23:27.085419 2228 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:23:27.085489 2228 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:27.085558 2228 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:23:27.085611 2228 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:27.094194 2228 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.130:46117
I20260808 18:23:27.094220 2362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.130:46117 every 8 connection(s)
I20260808 18:23:27.095125 2228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
I20260808 18:23:27.095799 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 2228
I20260808 18:23:27.095937 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance
I20260808 18:23:27.096235 23842 external_mini_cluster.cc:1428] Setting key 0552df585f0b916677ba579fb0631e14
I20260808 18:23:27.098385 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.131:0
--local_ip_for_outbound_sockets=127.23.72.131
--webserver_interface=127.23.72.131
--webserver_port=0
--tserver_master_addrs=127.23.72.190:44055
--builtin_ntp_servers=127.23.72.148:40127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 18:23:27.102474 2062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:27.097974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:35517 (local address 127.23.72.190:44055)
0808 18:23:27.098115 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:27.098119 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:27.099373 (+ 1254us) server_negotiation.cc:407] Connection header received
0808 18:23:27.100719 (+ 1346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:27.100722 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:27.100762 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:27.100837 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:27.101976 (+ 1139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:23:27.101979 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:23:27.101982 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:23:27.101994 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:23:27.102017 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:23:27.102029 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:23:27.102134 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:27.102279 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 18:23:27.102326 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 18:23:27.103315 2365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:27.098522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:44055 (local address 127.23.72.130:35517)
0808 18:23:27.099228 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:27.099268 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:27.100515 (+ 1247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:27.101001 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:27.101008 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:27.101098 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:27.101710 (+ 612us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:23:27.101712 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:27.101752 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:23:27.101819 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:23:27.102118 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:23:27.102126 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:23:27.102132 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 18:23:27.102312 (+ 180us) client_negotiation.cc:240] Negotiation successful
0808 18:23:27.102555 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":136,"threads_started":1}
I20260808 18:23:27.104506 2363 heartbeater.cc:347] Connected to a master server at 127.23.72.190:44055
I20260808 18:23:27.104791 2363 heartbeater.cc:464] Registering TS with master...
I20260808 18:23:27.105253 2363 heartbeater.cc:511] Master 127.23.72.190:44055 requested a full tablet report, sending...
I20260808 18:23:27.106385 2020 ts_manager.cc:194] Registered new tserver with Master: 06f7109e693c44bfaac7349d5e1fa5d1 (127.23.72.130:46117)
W20260808 18:23:27.211958 2366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:23:27.212189 2366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:23:27.212225 2366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:23:27.216094 2366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:23:27.216144 2366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:23:27.216197 2366 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.131
I20260808 18:23:27.220795 2366 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:40127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.72.131
--webserver_port=0
--tserver_master_addrs=127.23.72.190:44055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2366
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.131
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:23:27.221735 2366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:23:27.223006 2366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:23:27.232362 2373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:23:27.232390 2374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:23:27.232472 2366 server_base.cc:1034] running on GCE node
W20260808 18:23:27.232370 2376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:23:27.233166 2366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:23:27.233786 2366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:23:27.234936 2366 hybrid_clock.cc:648] HybridClock initialized: now 1786213407234903 us; error 45 us; skew 500 ppm
I20260808 18:23:27.236755 2366 webserver.cc:460] Webserver started at http://127.23.72.131:35617/ using document root <none> and password file <none>
I20260808 18:23:27.237342 2366 fs_manager.cc:359] Metadata directory not provided
I20260808 18:23:27.237404 2366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:23:27.237650 2366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:23:27.239729 2366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/data/instance:
uuid: "088334aa20c64a9fbe4b94892f3b7da4"
format_stamp: "Formatted at 2026-08-08 18:23:27 on dist-test-slave-5k54"
server_key: "200155971d217141aecd1fe3d0b42eb6"
server_key_iv: "64d713410a5abf202567ff54e2153869"
server_key_version: "encryptionkey@0"
I20260808 18:23:27.240285 2366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance:
uuid: "088334aa20c64a9fbe4b94892f3b7da4"
format_stamp: "Formatted at 2026-08-08 18:23:27 on dist-test-slave-5k54"
server_key: "200155971d217141aecd1fe3d0b42eb6"
server_key_iv: "64d713410a5abf202567ff54e2153869"
server_key_version: "encryptionkey@0"
I20260808 18:23:27.243800 2366 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 18:23:27.246296 2383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:27.247565 2366 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:23:27.247697 2366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/wal
uuid: "088334aa20c64a9fbe4b94892f3b7da4"
format_stamp: "Formatted at 2026-08-08 18:23:27 on dist-test-slave-5k54"
server_key: "200155971d217141aecd1fe3d0b42eb6"
server_key_iv: "64d713410a5abf202567ff54e2153869"
server_key_version: "encryptionkey@0"
I20260808 18:23:27.247808 2366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:23:27.252419 2366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:23:27.253011 2366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:23:27.253189 2366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:23:27.253705 2366 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:23:27.254566 2366 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:23:27.254637 2366 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:27.254701 2366 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:23:27.254755 2366 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:27.263710 2366 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.131:44935
I20260808 18:23:27.263736 2507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.131:44935 every 8 connection(s)
I20260808 18:23:27.264559 2366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
I20260808 18:23:27.265280 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 2366
I20260808 18:23:27.265391 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance
I20260808 18:23:27.265697 23842 external_mini_cluster.cc:1428] Setting key 0a2b7fbd370b5b6b84e735c9fa9e049c
I20260808 18:23:27.270922 2062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:27.267895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:47705 (local address 127.23.72.190:44055)
0808 18:23:27.268060 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:27.268065 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:27.268082 (+ 17us) server_negotiation.cc:407] Connection header received
0808 18:23:27.269309 (+ 1227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:27.269312 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:27.269363 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:27.269439 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:27.270432 (+ 993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:23:27.270435 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:23:27.270438 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:23:27.270450 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:23:27.270471 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:23:27.270479 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:23:27.270577 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:27.270749 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 18:23:27.270795 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 18:23:27.271874 2510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:27.266867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:44055 (local address 127.23.72.131:47705)
0808 18:23:27.267769 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:27.267824 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:27.269110 (+ 1286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:27.269597 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:27.269603 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:27.269687 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:27.270191 (+ 504us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:23:27.270192 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:27.270231 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:23:27.270294 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:23:27.270577 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:23:27.270586 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:23:27.270594 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 18:23:27.270779 (+ 185us) client_negotiation.cc:240] Negotiation successful
0808 18:23:27.271063 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":125,"threads_started":1}
I20260808 18:23:27.272955 2508 heartbeater.cc:347] Connected to a master server at 127.23.72.190:44055
I20260808 18:23:27.273188 2508 heartbeater.cc:464] Registering TS with master...
I20260808 18:23:27.273577 2508 heartbeater.cc:511] Master 127.23.72.190:44055 requested a full tablet report, sending...
I20260808 18:23:27.274561 2020 ts_manager.cc:194] Registered new tserver with Master: 088334aa20c64a9fbe4b94892f3b7da4 (127.23.72.131:44935)
I20260808 18:23:27.279517 23842 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 18:23:27.282282 2062 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:23:27.281486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58496 (local address 127.23.72.190:44055)
0808 18:23:27.281610 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:27.281614 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:27.281768 (+ 154us) server_negotiation.cc:407] Connection header received
0808 18:23:27.281845 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:27.281847 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:27.281883 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:27.281945 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:27.282158 (+ 213us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58496: BlockingRecv error: recv got EOF from 127.0.0.1:58496 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 18:23:27.282415 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 2085
I20260808 18:23:27.291815 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 2228
I20260808 18:23:27.297400 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 2366
I20260808 18:23:27.302953 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 1988
2026-08-08T18:23:27Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (815 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:23:27 dist-test-slave-5k54 krb5kdc[2518](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:23:27 dist-test-slave-5k54 krb5kdc[2518](info): set up 2 sockets
Aug 08 18:23:27 dist-test-slave-5k54 krb5kdc[2518](info): commencing operation
krb5kdc: starting...
W20260808 18:23:28.393887 23842 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Aug 08 18:23:28 dist-test-slave-5k54 krb5kdc[2518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213408, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:23:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:23:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.72.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
I20260808 18:23:28.527563 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:43027
--webserver_interface=127.23.72.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:36869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:43027
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:23:28.641716 2532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:23:28.642107 2532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:23:28.642174 2532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:23:28.646476 2532 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:23:28.646533 2532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:23:28.646552 2532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:23:28.646569 2532 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:23:28.646627 2532 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:23:28.651957 2532 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:36869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:43027
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:43027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.72.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2532
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:23:28.653033 2532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:23:28.654065 2532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:23:28.662575 2540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:23:28.662539 2537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:23:28.662550 2538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:23:28.662750 2532 server_base.cc:1034] running on GCE node
I20260808 18:23:28.663411 2532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:23:28.664477 2532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:23:28.665633 2532 hybrid_clock.cc:648] HybridClock initialized: now 1786213408665611 us; error 33 us; skew 500 ppm
Aug 08 18:23:28 dist-test-slave-5k54 krb5kdc[2518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213408, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:23:28.673278 2532 init.cc:377] Logged in from keytab as kudu/127.23.72.190@KRBTEST.COM (short username kudu)
I20260808 18:23:28.674257 2532 webserver.cc:460] Webserver started at http://127.23.72.190:41711/ using document root <none> and password file <none>
I20260808 18:23:28.674823 2532 fs_manager.cc:359] Metadata directory not provided
I20260808 18:23:28.674887 2532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:23:28.675122 2532 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:23:28.677254 2532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "e317d7552be947ecaab3bcc854b04f59"
format_stamp: "Formatted at 2026-08-08 18:23:28 on dist-test-slave-5k54"
server_key: "9c8c934c5528f18d406310d76a5277e9"
server_key_iv: "2830f6c1bf3fe3b05ce04f3b45997c5f"
server_key_version: "encryptionkey@0"
I20260808 18:23:28.677906 2532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "e317d7552be947ecaab3bcc854b04f59"
format_stamp: "Formatted at 2026-08-08 18:23:28 on dist-test-slave-5k54"
server_key: "9c8c934c5528f18d406310d76a5277e9"
server_key_iv: "2830f6c1bf3fe3b05ce04f3b45997c5f"
server_key_version: "encryptionkey@0"
I20260808 18:23:28.681634 2532 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 18:23:28.684232 2547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:28.685592 2532 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 18:23:28.685734 2532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "e317d7552be947ecaab3bcc854b04f59"
format_stamp: "Formatted at 2026-08-08 18:23:28 on dist-test-slave-5k54"
server_key: "9c8c934c5528f18d406310d76a5277e9"
server_key_iv: "2830f6c1bf3fe3b05ce04f3b45997c5f"
server_key_version: "encryptionkey@0"
I20260808 18:23:28.685845 2532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:23:28.699216 2532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:23:28.702392 2532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:23:28.702596 2532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:23:28.710202 2532 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.190:43027
I20260808 18:23:28.710253 2603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.190:43027 every 8 connection(s)
I20260808 18:23:28.711179 2532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:23:28.713907 2604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:23:28.714972 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 2532
I20260808 18:23:28.715119 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:23:28.715405 23842 external_mini_cluster.cc:1428] Setting key b6a6b9667f02dba76a493afd40785dc3
I20260808 18:23:28.720628 2604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e317d7552be947ecaab3bcc854b04f59: Bootstrap starting.
I20260808 18:23:28.723227 2604 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e317d7552be947ecaab3bcc854b04f59: Neither blocks nor log segments found. Creating new log.
I20260808 18:23:28.724022 2604 log.cc:824] T 00000000000000000000000000000000 P e317d7552be947ecaab3bcc854b04f59: Log is configured to *not* fsync() on all Append() calls
Aug 08 18:23:28 dist-test-slave-5k54 krb5kdc[2518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213408, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:23:28.726289 2604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e317d7552be947ecaab3bcc854b04f59: No bootstrap required, opened a new log
I20260808 18:23:28.730558 2607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:28.716838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58318 (local address 127.23.72.190:43027)
0808 18:23:28.717277 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:28.717297 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:28.717341 (+ 44us) server_negotiation.cc:407] Connection header received
0808 18:23:28.718166 (+ 825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:28.718190 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:28.718608 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:28.718988 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:28.719687 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:28.721085 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:28.721904 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:28.722555 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:28.722715 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:28.725361 (+ 2646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:23:28.725367 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:23:28.725382 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:23:28.725411 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:23:28.727328 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:23:28.727812 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:23:28.727818 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:23:28.727824 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:23:28.727968 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:23:28.728252 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:23:28.728256 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:23:28.728257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:23:28.728849 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:23:28.729058 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:28.729272 (+ 214us) server_negotiation.cc:299] Negotiation successful
0808 18:23:28.729571 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":105,"threads_started":1}
I20260808 18:23:28.730909 2604 raft_consensus.cc:348] T 00000000000000000000000000000000 P e317d7552be947ecaab3bcc854b04f59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e317d7552be947ecaab3bcc854b04f59" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 43027 } }
I20260808 18:23:28.731130 2604 raft_consensus.cc:374] T 00000000000000000000000000000000 P e317d7552be947ecaab3bcc854b04f59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:23:28.731202 2604 raft_consensus.cc:729] T 00000000000000000000000000000000 P e317d7552be947ecaab3bcc854b04f59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e317d7552be947ecaab3bcc854b04f59, State: Initialized, Role: FOLLOWER
I20260808 18:23:28.731675 2604 consensus_queue.cc:260] T 00000000000000000000000000000000 P e317d7552be947ecaab3bcc854b04f59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e317d7552be947ecaab3bcc854b04f59" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 43027 } }
I20260808 18:23:28.731830 2604 raft_consensus.cc:388] T 00000000000000000000000000000000 P e317d7552be947ecaab3bcc854b04f59 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:23:28.731925 2604 raft_consensus.cc:482] T 00000000000000000000000000000000 P e317d7552be947ecaab3bcc854b04f59 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:23:28.732026 2604 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e317d7552be947ecaab3bcc854b04f59 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:23:28.732975 2604 raft_consensus.cc:504] T 00000000000000000000000000000000 P e317d7552be947ecaab3bcc854b04f59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e317d7552be947ecaab3bcc854b04f59" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 43027 } }
I20260808 18:23:28.733316 2604 leader_election.cc:302] T 00000000000000000000000000000000 P e317d7552be947ecaab3bcc854b04f59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e317d7552be947ecaab3bcc854b04f59; no voters:
I20260808 18:23:28.733625 2604 leader_election.cc:288] T 00000000000000000000000000000000 P e317d7552be947ecaab3bcc854b04f59 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:23:28.733724 2609 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e317d7552be947ecaab3bcc854b04f59 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:23:28.734004 2609 raft_consensus.cc:686] T 00000000000000000000000000000000 P e317d7552be947ecaab3bcc854b04f59 [term 1 LEADER]: Becoming Leader. State: Replica: e317d7552be947ecaab3bcc854b04f59, State: Running, Role: LEADER
I20260808 18:23:28.734301 2609 consensus_queue.cc:237] T 00000000000000000000000000000000 P e317d7552be947ecaab3bcc854b04f59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e317d7552be947ecaab3bcc854b04f59" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 43027 } }
I20260808 18:23:28.734910 2604 sys_catalog.cc:564] T 00000000000000000000000000000000 P e317d7552be947ecaab3bcc854b04f59 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:23:28.736007 2610 sys_catalog.cc:455] T 00000000000000000000000000000000 P e317d7552be947ecaab3bcc854b04f59 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e317d7552be947ecaab3bcc854b04f59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e317d7552be947ecaab3bcc854b04f59" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 43027 } } }
I20260808 18:23:28.735986 2611 sys_catalog.cc:455] T 00000000000000000000000000000000 P e317d7552be947ecaab3bcc854b04f59 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e317d7552be947ecaab3bcc854b04f59. Latest consensus state: current_term: 1 leader_uuid: "e317d7552be947ecaab3bcc854b04f59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e317d7552be947ecaab3bcc854b04f59" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 43027 } } }
I20260808 18:23:28.736173 2611 sys_catalog.cc:458] T 00000000000000000000000000000000 P e317d7552be947ecaab3bcc854b04f59 [sys.catalog]: This master's current role is: LEADER
I20260808 18:23:28.736173 2610 sys_catalog.cc:458] T 00000000000000000000000000000000 P e317d7552be947ecaab3bcc854b04f59 [sys.catalog]: This master's current role is: LEADER
I20260808 18:23:28.736824 2619 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:23:28.739610 2619 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:23:28.745065 2619 catalog_manager.cc:1347] Generated new cluster ID: ae2841d1d7b3445681c140b0b718b56e
I20260808 18:23:28.745141 2619 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:23:28.753794 2619 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:23:28.754662 2619 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:23:28.767045 2619 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e317d7552be947ecaab3bcc854b04f59: Generated new TSK 0
I20260808 18:23:28.767913 2619 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.72.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab.
Entry for principal kudu/127.23.72.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab.
I20260808 18:23:28.806727 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.129:0
--local_ip_for_outbound_sockets=127.23.72.129
--webserver_interface=127.23.72.129
--webserver_port=0
--tserver_master_addrs=127.23.72.190:43027
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab
--principal=kudu/127.23.72.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:36869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:23:28.927973 2632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:23:28.928246 2632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:23:28.928334 2632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:23:28.932287 2632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:23:28.932380 2632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:23:28.932472 2632 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.129
I20260808 18:23:28.938024 2632 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:36869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab
--principal=kudu/127.23.72.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.72.129
--webserver_port=0
--tserver_master_addrs=127.23.72.190:43027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2632
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.129
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:23:28.939057 2632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:23:28.940079 2632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:23:28.949805 2640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:23:28.949973 2632 server_base.cc:1034] running on GCE node
W20260808 18:23:28.949837 2637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:23:28.950126 2638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:23:28.950526 2632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:23:28.951155 2632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:23:28.952312 2632 hybrid_clock.cc:648] HybridClock initialized: now 1786213408952281 us; error 49 us; skew 500 ppm
Aug 08 18:23:28 dist-test-slave-5k54 krb5kdc[2518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213408, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:23:28.955180 2632 init.cc:377] Logged in from keytab as kudu/127.23.72.129@KRBTEST.COM (short username kudu)
I20260808 18:23:28.956110 2632 webserver.cc:460] Webserver started at http://127.23.72.129:46837/ using document root <none> and password file <none>
I20260808 18:23:28.956693 2632 fs_manager.cc:359] Metadata directory not provided
I20260808 18:23:28.956786 2632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:23:28.957018 2632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:23:28.959071 2632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/data/instance:
uuid: "eff72d376a4a460c948075a561a4a611"
format_stamp: "Formatted at 2026-08-08 18:23:28 on dist-test-slave-5k54"
server_key: "2b9c9f0429acfce9c9e569c6026da63c"
server_key_iv: "ed8ea387441b6f3e80493df6b77a4540"
server_key_version: "encryptionkey@0"
I20260808 18:23:28.959707 2632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance:
uuid: "eff72d376a4a460c948075a561a4a611"
format_stamp: "Formatted at 2026-08-08 18:23:28 on dist-test-slave-5k54"
server_key: "2b9c9f0429acfce9c9e569c6026da63c"
server_key_iv: "ed8ea387441b6f3e80493df6b77a4540"
server_key_version: "encryptionkey@0"
I20260808 18:23:28.963523 2632 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260808 18:23:28.966033 2648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:28.967262 2632 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 18:23:28.967407 2632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/wal
uuid: "eff72d376a4a460c948075a561a4a611"
format_stamp: "Formatted at 2026-08-08 18:23:28 on dist-test-slave-5k54"
server_key: "2b9c9f0429acfce9c9e569c6026da63c"
server_key_iv: "ed8ea387441b6f3e80493df6b77a4540"
server_key_version: "encryptionkey@0"
I20260808 18:23:28.967581 2632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:23:28.988864 2632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:23:28.991983 2632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:23:28.992182 2632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:23:28.992802 2632 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:23:28.993618 2632 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:23:28.993686 2632 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:28.993758 2632 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:23:28.993816 2632 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:29.002588 2632 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.129:41047
I20260808 18:23:29.002617 2772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.129:41047 every 8 connection(s)
I20260808 18:23:29.003540 2632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
I20260808 18:23:29.004261 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 2632
I20260808 18:23:29.004390 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance
I20260808 18:23:29.004822 23842 external_mini_cluster.cc:1428] Setting key 01b6b52e0386d6c3e3cf43ec28478c16
Aug 08 18:23:29 dist-test-slave-5k54 krb5kdc[2518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213408, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.72.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.130@KRBTEST.COM" created.
I20260808 18:23:29.019565 2607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:29.005666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:52871 (local address 127.23.72.190:43027)
0808 18:23:29.005826 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:29.005832 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:29.006851 (+ 1019us) server_negotiation.cc:407] Connection header received
0808 18:23:29.008331 (+ 1480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:29.008337 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:29.008408 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:29.008509 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:29.010197 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:29.011069 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:29.012298 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:29.012738 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:29.012873 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:29.015602 (+ 2729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:23:29.015608 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:23:29.015612 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:23:29.015645 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:23:29.017108 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:23:29.017727 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:23:29.017729 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:23:29.017731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:23:29.017807 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:23:29.018232 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:23:29.018235 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:23:29.018236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:23:29.018488 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:23:29.018568 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:29.019137 (+ 569us) server_negotiation.cc:299] Negotiation successful
0808 18:23:29.019298 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 18:23:29.020473 2775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:29.005995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:43027 (local address 127.23.72.129:52871)
0808 18:23:29.006648 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:29.006712 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:29.008098 (+ 1386us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:29.008730 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:29.008742 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:29.009288 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:29.010025 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:29.010046 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:29.011208 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:29.011211 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:29.012175 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:29.012185 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:29.012906 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:29.012909 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:29.013046 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:29.013667 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:29.013683 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:29.015389 (+ 1706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:23:29.017313 (+ 1924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:23:29.017322 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:23:29.017338 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:23:29.017617 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:23:29.017914 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:23:29.017917 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:23:29.017918 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:23:29.018060 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:23:29.018606 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:23:29.018611 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:23:29.018828 (+ 217us) client_negotiation.cc:769] Sending connection context
0808 18:23:29.019035 (+ 207us) client_negotiation.cc:240] Negotiation successful
0808 18:23:29.019341 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":150,"threads_started":1}
I20260808 18:23:29.021723 2773 heartbeater.cc:347] Connected to a master server at 127.23.72.190:43027
I20260808 18:23:29.021999 2773 heartbeater.cc:464] Registering TS with master...
I20260808 18:23:29.022591 2773 heartbeater.cc:511] Master 127.23.72.190:43027 requested a full tablet report, sending...
I20260808 18:23:29.024206 2566 ts_manager.cc:194] Registered new tserver with Master: eff72d376a4a460c948075a561a4a611 (127.23.72.129:41047)
I20260808 18:23:29.025372 2566 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.129@KRBTEST.COM'} at 127.23.72.129:52871
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab.
Entry for principal kudu/127.23.72.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab.
I20260808 18:23:29.036669 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.130:0
--local_ip_for_outbound_sockets=127.23.72.130
--webserver_interface=127.23.72.130
--webserver_port=0
--tserver_master_addrs=127.23.72.190:43027
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab
--principal=kudu/127.23.72.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:36869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:23:29.151698 2779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:23:29.151947 2779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:23:29.152020 2779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:23:29.155992 2779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:23:29.156046 2779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:23:29.156126 2779 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.130
I20260808 18:23:29.161262 2779 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:36869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab
--principal=kudu/127.23.72.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.72.130
--webserver_port=0
--tserver_master_addrs=127.23.72.190:43027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2779
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.130
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:23:29.162312 2779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:23:29.163367 2779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:23:29.172748 2790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:23:29.172796 2786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:23:29.172763 2785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:23:29.173056 2779 server_base.cc:1034] running on GCE node
I20260808 18:23:29.173448 2779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:23:29.174041 2779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:23:29.175196 2779 hybrid_clock.cc:648] HybridClock initialized: now 1786213409175150 us; error 57 us; skew 500 ppm
Aug 08 18:23:29 dist-test-slave-5k54 krb5kdc[2518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213409, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:23:29.177958 2779 init.cc:377] Logged in from keytab as kudu/127.23.72.130@KRBTEST.COM (short username kudu)
I20260808 18:23:29.178748 2779 webserver.cc:460] Webserver started at http://127.23.72.130:39925/ using document root <none> and password file <none>
I20260808 18:23:29.179255 2779 fs_manager.cc:359] Metadata directory not provided
I20260808 18:23:29.179311 2779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:23:29.179541 2779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:23:29.181481 2779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/data/instance:
uuid: "d24fb014a4d64e49b1a8e9f4038570e7"
format_stamp: "Formatted at 2026-08-08 18:23:29 on dist-test-slave-5k54"
server_key: "c9fd6fe7bc486b45744861bf88754e7e"
server_key_iv: "046a679964a9fbab174f74d7e5d0e596"
server_key_version: "encryptionkey@0"
I20260808 18:23:29.181998 2779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance:
uuid: "d24fb014a4d64e49b1a8e9f4038570e7"
format_stamp: "Formatted at 2026-08-08 18:23:29 on dist-test-slave-5k54"
server_key: "c9fd6fe7bc486b45744861bf88754e7e"
server_key_iv: "046a679964a9fbab174f74d7e5d0e596"
server_key_version: "encryptionkey@0"
I20260808 18:23:29.185782 2779 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 18:23:29.188200 2797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:29.189556 2779 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:23:29.189678 2779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/wal
uuid: "d24fb014a4d64e49b1a8e9f4038570e7"
format_stamp: "Formatted at 2026-08-08 18:23:29 on dist-test-slave-5k54"
server_key: "c9fd6fe7bc486b45744861bf88754e7e"
server_key_iv: "046a679964a9fbab174f74d7e5d0e596"
server_key_version: "encryptionkey@0"
I20260808 18:23:29.189785 2779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:23:29.199309 2779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:23:29.202147 2779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:23:29.202335 2779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:23:29.202883 2779 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:23:29.203732 2779 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:23:29.203800 2779 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:29.203864 2779 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:23:29.203907 2779 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:29.212476 2779 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.130:38007
I20260808 18:23:29.212515 2923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.130:38007 every 8 connection(s)
I20260808 18:23:29.213325 2779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
Aug 08 18:23:29 dist-test-slave-5k54 krb5kdc[2518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213409, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.130@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:23:29.223464 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 2779
I20260808 18:23:29.223623 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance
I20260808 18:23:29.223937 23842 external_mini_cluster.cc:1428] Setting key e3d745cd9662416f5e624b95a25f6454
I20260808 18:23:29.227402 2607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:29.215027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:44043 (local address 127.23.72.190:43027)
0808 18:23:29.215164 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:29.215168 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:29.216038 (+ 870us) server_negotiation.cc:407] Connection header received
0808 18:23:29.217464 (+ 1426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:29.217467 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:29.217530 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:29.217613 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:29.218766 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:29.219316 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:29.220483 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:29.220718 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:29.220844 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:29.223375 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:23:29.223378 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:23:29.223380 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:23:29.223405 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:23:29.225194 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:23:29.225695 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:23:29.225698 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:23:29.225701 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:23:29.225795 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:23:29.226130 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:23:29.226136 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:23:29.226141 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:23:29.226436 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:23:29.226546 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:29.227003 (+ 457us) server_negotiation.cc:299] Negotiation successful
0808 18:23:29.227145 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 18:23:29.228125 2926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:29.215388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:43027 (local address 127.23.72.130:44043)
0808 18:23:29.215895 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:29.215936 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:29.217230 (+ 1294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:29.217780 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:29.217787 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:29.218149 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:29.218618 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:29.218630 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:29.219466 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:29.219476 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:29.220355 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:29.220364 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:29.220876 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:29.220880 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:29.221050 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:29.221673 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:29.221689 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:29.223195 (+ 1506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:23:29.225317 (+ 2122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:23:29.225322 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:23:29.225334 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:23:29.225591 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:23:29.225888 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:23:29.225890 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:23:29.225891 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:23:29.226036 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:23:29.226540 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:23:29.226545 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:23:29.226770 (+ 225us) client_negotiation.cc:769] Sending connection context
0808 18:23:29.226925 (+ 155us) client_negotiation.cc:240] Negotiation successful
0808 18:23:29.227192 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":118,"threads_started":1}
I20260808 18:23:29.229298 2924 heartbeater.cc:347] Connected to a master server at 127.23.72.190:43027
I20260808 18:23:29.229559 2924 heartbeater.cc:464] Registering TS with master...
I20260808 18:23:29.230043 2924 heartbeater.cc:511] Master 127.23.72.190:43027 requested a full tablet report, sending...
I20260808 18:23:29.231101 2566 ts_manager.cc:194] Registered new tserver with Master: d24fb014a4d64e49b1a8e9f4038570e7 (127.23.72.130:38007)
I20260808 18:23:29.232049 2566 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.130@KRBTEST.COM'} at 127.23.72.130:44043
WARNING: no policy specified for kudu/127.23.72.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab.
Entry for principal kudu/127.23.72.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab.
I20260808 18:23:29.254318 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.131:0
--local_ip_for_outbound_sockets=127.23.72.131
--webserver_interface=127.23.72.131
--webserver_port=0
--tserver_master_addrs=127.23.72.190:43027
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab
--principal=kudu/127.23.72.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:36869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:23:29.366940 2929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:23:29.367184 2929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:23:29.367225 2929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:23:29.371591 2929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:23:29.371644 2929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:23:29.371733 2929 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.131
I20260808 18:23:29.376848 2929 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:36869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab
--principal=kudu/127.23.72.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.72.131
--webserver_port=0
--tserver_master_addrs=127.23.72.190:43027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2929
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.131
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:23:29.377811 2929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:23:29.378779 2929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:23:29.388211 2935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:23:29.388217 2934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:23:29.388217 2939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:23:29.388614 2929 server_base.cc:1034] running on GCE node
I20260808 18:23:29.389073 2929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:23:29.389744 2929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:23:29.390949 2929 hybrid_clock.cc:648] HybridClock initialized: now 1786213409390916 us; error 33 us; skew 500 ppm
Aug 08 18:23:29 dist-test-slave-5k54 krb5kdc[2518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213409, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:23:29.393909 2929 init.cc:377] Logged in from keytab as kudu/127.23.72.131@KRBTEST.COM (short username kudu)
I20260808 18:23:29.394788 2929 webserver.cc:460] Webserver started at http://127.23.72.131:39829/ using document root <none> and password file <none>
I20260808 18:23:29.395340 2929 fs_manager.cc:359] Metadata directory not provided
I20260808 18:23:29.395471 2929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:23:29.395726 2929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:23:29.397771 2929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/data/instance:
uuid: "e431ba7482f2472eb0272babad420a55"
format_stamp: "Formatted at 2026-08-08 18:23:29 on dist-test-slave-5k54"
server_key: "e9098d8193b8857cd282ac40deff57d9"
server_key_iv: "c34fe3d73a8df4912ac07d4d46ba05d2"
server_key_version: "encryptionkey@0"
I20260808 18:23:29.398346 2929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance:
uuid: "e431ba7482f2472eb0272babad420a55"
format_stamp: "Formatted at 2026-08-08 18:23:29 on dist-test-slave-5k54"
server_key: "e9098d8193b8857cd282ac40deff57d9"
server_key_iv: "c34fe3d73a8df4912ac07d4d46ba05d2"
server_key_version: "encryptionkey@0"
I20260808 18:23:29.402071 2929 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.002s
I20260808 18:23:29.404448 2946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:29.405586 2929 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:23:29.405714 2929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/wal
uuid: "e431ba7482f2472eb0272babad420a55"
format_stamp: "Formatted at 2026-08-08 18:23:29 on dist-test-slave-5k54"
server_key: "e9098d8193b8857cd282ac40deff57d9"
server_key_iv: "c34fe3d73a8df4912ac07d4d46ba05d2"
server_key_version: "encryptionkey@0"
I20260808 18:23:29.405821 2929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:23:29.416817 2929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:23:29.420218 2929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:23:29.420521 2929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:23:29.421175 2929 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:23:29.421995 2929 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:23:29.422067 2929 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:29.422137 2929 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:23:29.422189 2929 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:23:29.431713 2929 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.131:36443
I20260808 18:23:29.431733 3073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.131:36443 every 8 connection(s)
I20260808 18:23:29.432682 2929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
I20260808 18:23:29.441305 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 2929
I20260808 18:23:29.441457 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance
I20260808 18:23:29.441768 23842 external_mini_cluster.cc:1428] Setting key c323a7abb992af56f8a8866af4d57df3
Aug 08 18:23:29 dist-test-slave-5k54 krb5kdc[2518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213409, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.131@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:23:29.446823 2607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:29.434550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:38891 (local address 127.23.72.190:43027)
0808 18:23:29.434693 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:29.434697 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:29.435616 (+ 919us) server_negotiation.cc:407] Connection header received
0808 18:23:29.437259 (+ 1643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:29.437262 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:29.437324 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:29.437423 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:29.438610 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:29.439154 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:29.440357 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:29.440584 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:29.440696 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:29.443324 (+ 2628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:23:29.443328 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:23:29.443333 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:23:29.443366 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:23:29.444754 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:23:29.445337 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:23:29.445340 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:23:29.445342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:23:29.445425 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:23:29.445758 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:23:29.445761 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:23:29.445763 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:23:29.445948 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:23:29.446056 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:29.446538 (+ 482us) server_negotiation.cc:299] Negotiation successful
0808 18:23:29.446655 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 18:23:29.447664 3076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:29.434913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:43027 (local address 127.23.72.131:38891)
0808 18:23:29.435432 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:29.435493 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:29.437015 (+ 1522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:29.437600 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:29.437608 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:29.437978 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:23:29.438456 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:29.438469 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:29.439281 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:29.439291 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:29.440230 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:29.440240 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:29.440740 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:29.440744 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:29.440919 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:29.441631 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:23:29.441648 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:23:29.443145 (+ 1497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:23:29.444938 (+ 1793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:23:29.444944 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:23:29.444959 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:23:29.445228 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:23:29.445519 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:23:29.445522 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:23:29.445524 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:23:29.445660 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:23:29.446054 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:23:29.446061 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:23:29.446299 (+ 238us) client_negotiation.cc:769] Sending connection context
0808 18:23:29.446496 (+ 197us) client_negotiation.cc:240] Negotiation successful
0808 18:23:29.446803 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":133,"threads_started":1}
I20260808 18:23:29.448702 3074 heartbeater.cc:347] Connected to a master server at 127.23.72.190:43027
I20260808 18:23:29.448921 3074 heartbeater.cc:464] Registering TS with master...
I20260808 18:23:29.449365 3074 heartbeater.cc:511] Master 127.23.72.190:43027 requested a full tablet report, sending...
I20260808 18:23:29.450282 2566 ts_manager.cc:194] Registered new tserver with Master: e431ba7482f2472eb0272babad420a55 (127.23.72.131:36443)
I20260808 18:23:29.450795 2566 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.131@KRBTEST.COM'} at 127.23.72.131:38891
I20260808 18:23:29.455746 23842 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:23:29.464993 2607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:29.457644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58320 (local address 127.23.72.190:43027)
0808 18:23:29.457785 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:29.457790 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:29.457953 (+ 163us) server_negotiation.cc:407] Connection header received
0808 18:23:29.458054 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:29.458057 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:29.458094 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:29.458155 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:23:29.458858 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:29.459368 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:29.460319 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:29.460525 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:29.460606 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:29.461916 (+ 1310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:23:29.461919 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:23:29.461921 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:23:29.461943 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:23:29.463230 (+ 1287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:23:29.463664 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:23:29.463667 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:23:29.463668 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:23:29.463749 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:23:29.464051 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:23:29.464053 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:23:29.464056 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:23:29.464301 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:23:29.464443 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:29.464627 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 18:23:29.464831 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 18:23:29.468031 2566 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58320:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:23:29.471186 2566 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:23:29.485678 3090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:29.480117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:41047 (local address 127.0.0.1:49710)
0808 18:23:29.480928 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:29.480939 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:29.481014 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:29.481467 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:29.481469 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:29.481486 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:23:29.481588 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:29.481598 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:29.482864 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:29.482867 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:29.483971 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:29.483981 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:29.485111 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:29.485115 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:29.485225 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:29.485295 (+ 70us) client_negotiation.cc:769] Sending connection context
0808 18:23:29.485398 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 18:23:29.485528 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":762,"mutex_wait_us":41,"thread_start_us":89,"threads_started":1}
I20260808 18:23:29.485849 3091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:29.480583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49710 (local address 127.23.72.129:41047)
0808 18:23:29.480969 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:29.480984 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:29.481006 (+ 22us) server_negotiation.cc:407] Connection header received
0808 18:23:29.481160 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:29.481164 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:29.481325 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:29.481477 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:23:29.481709 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:29.482742 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:29.484095 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:29.484975 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:29.485095 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:29.485234 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:29.485563 (+ 329us) server_negotiation.cc:299] Negotiation successful
0808 18:23:29.485656 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":108,"threads_started":1}
I20260808 18:23:29.485929 3092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:29.480556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50270 (local address 127.23.72.130:38007)
0808 18:23:29.481199 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:29.481212 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:29.481235 (+ 23us) server_negotiation.cc:407] Connection header received
0808 18:23:29.481324 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:29.481335 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:29.481513 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:29.481917 (+ 404us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:23:29.482047 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:29.482963 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:29.484419 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:29.485217 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:29.485329 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:29.485473 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:29.485687 (+ 214us) server_negotiation.cc:299] Negotiation successful
0808 18:23:29.485783 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":137,"threads_started":1}
I20260808 18:23:29.485860 3086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:29.479481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.130:38007 (local address 127.0.0.1:50270)
0808 18:23:29.480290 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:29.480303 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:29.480388 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:29.481686 (+ 1298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:29.481688 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:29.481708 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:23:29.481800 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:29.481804 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:29.483114 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:29.483117 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:29.484297 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:29.484305 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:29.485315 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:29.485318 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:29.485469 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:29.485534 (+ 65us) client_negotiation.cc:769] Sending connection context
0808 18:23:29.485602 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 18:23:29.485691 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":720,"mutex_wait_us":79,"thread_start_us":107,"threads_started":1}
I20260808 18:23:29.486207 3089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:29.479547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55486 (local address 127.23.72.131:36443)
0808 18:23:29.479992 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:29.480006 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:29.480307 (+ 301us) server_negotiation.cc:407] Connection header received
0808 18:23:29.480393 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:29.480398 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:29.480617 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:29.480784 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:23:29.481303 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:29.482530 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:29.484251 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:29.485009 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:29.485101 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:29.485234 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:29.485950 (+ 716us) server_negotiation.cc:299] Negotiation successful
0808 18:23:29.486061 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":154,"threads_started":1}
I20260808 18:23:29.486589 3087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:29.479367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.131:36443 (local address 127.0.0.1:55486)
0808 18:23:29.479948 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:29.479999 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:29.480138 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:29.480994 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:29.481001 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:29.481027 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:23:29.481177 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:29.481184 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:29.482671 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:29.482674 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:29.484123 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:29.484135 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:29.485303 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:29.485306 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:29.485492 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:29.485530 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 18:23:29.485602 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 18:23:29.486456 (+ 854us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"mutex_wait_us":35,"thread_start_us":108,"threads_started":1}
I20260808 18:23:29.489174 2847 tablet_service.cc:1467] Processing CreateTablet for tablet 6fce04ac2f644286b90a04ece609f93a (DEFAULT_TABLE table=test-table [id=bad1a19658974b7d84cffeb8ddefe425]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:23:29.489228 2703 tablet_service.cc:1467] Processing CreateTablet for tablet 6fce04ac2f644286b90a04ece609f93a (DEFAULT_TABLE table=test-table [id=bad1a19658974b7d84cffeb8ddefe425]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:23:29.489790 3001 tablet_service.cc:1467] Processing CreateTablet for tablet 6fce04ac2f644286b90a04ece609f93a (DEFAULT_TABLE table=test-table [id=bad1a19658974b7d84cffeb8ddefe425]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:23:29.490325 2847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fce04ac2f644286b90a04ece609f93a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:23:29.490326 2703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fce04ac2f644286b90a04ece609f93a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:23:29.490677 3001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fce04ac2f644286b90a04ece609f93a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:23:29.496639 3093 tablet_bootstrap.cc:492] T 6fce04ac2f644286b90a04ece609f93a P d24fb014a4d64e49b1a8e9f4038570e7: Bootstrap starting.
I20260808 18:23:29.497800 3094 tablet_bootstrap.cc:492] T 6fce04ac2f644286b90a04ece609f93a P eff72d376a4a460c948075a561a4a611: Bootstrap starting.
I20260808 18:23:29.498328 3096 tablet_bootstrap.cc:492] T 6fce04ac2f644286b90a04ece609f93a P e431ba7482f2472eb0272babad420a55: Bootstrap starting.
I20260808 18:23:29.499226 3093 tablet_bootstrap.cc:654] T 6fce04ac2f644286b90a04ece609f93a P d24fb014a4d64e49b1a8e9f4038570e7: Neither blocks nor log segments found. Creating new log.
I20260808 18:23:29.500008 3094 tablet_bootstrap.cc:654] T 6fce04ac2f644286b90a04ece609f93a P eff72d376a4a460c948075a561a4a611: Neither blocks nor log segments found. Creating new log.
I20260808 18:23:29.500021 3093 log.cc:824] T 6fce04ac2f644286b90a04ece609f93a P d24fb014a4d64e49b1a8e9f4038570e7: Log is configured to *not* fsync() on all Append() calls
I20260808 18:23:29.500451 3096 tablet_bootstrap.cc:654] T 6fce04ac2f644286b90a04ece609f93a P e431ba7482f2472eb0272babad420a55: Neither blocks nor log segments found. Creating new log.
I20260808 18:23:29.501320 3094 log.cc:824] T 6fce04ac2f644286b90a04ece609f93a P eff72d376a4a460c948075a561a4a611: Log is configured to *not* fsync() on all Append() calls
I20260808 18:23:29.501628 3096 log.cc:824] T 6fce04ac2f644286b90a04ece609f93a P e431ba7482f2472eb0272babad420a55: Log is configured to *not* fsync() on all Append() calls
I20260808 18:23:29.502666 3093 tablet_bootstrap.cc:492] T 6fce04ac2f644286b90a04ece609f93a P d24fb014a4d64e49b1a8e9f4038570e7: No bootstrap required, opened a new log
I20260808 18:23:29.502869 3093 ts_tablet_manager.cc:1397] T 6fce04ac2f644286b90a04ece609f93a P d24fb014a4d64e49b1a8e9f4038570e7: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260808 18:23:29.503355 3094 tablet_bootstrap.cc:492] T 6fce04ac2f644286b90a04ece609f93a P eff72d376a4a460c948075a561a4a611: No bootstrap required, opened a new log
I20260808 18:23:29.503585 3094 ts_tablet_manager.cc:1397] T 6fce04ac2f644286b90a04ece609f93a P eff72d376a4a460c948075a561a4a611: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 18:23:29.503674 3096 tablet_bootstrap.cc:492] T 6fce04ac2f644286b90a04ece609f93a P e431ba7482f2472eb0272babad420a55: No bootstrap required, opened a new log
I20260808 18:23:29.503850 3096 ts_tablet_manager.cc:1397] T 6fce04ac2f644286b90a04ece609f93a P e431ba7482f2472eb0272babad420a55: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 18:23:29.506541 3094 raft_consensus.cc:348] T 6fce04ac2f644286b90a04ece609f93a P eff72d376a4a460c948075a561a4a611 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e431ba7482f2472eb0272babad420a55" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 36443 } } peers { permanent_uuid: "d24fb014a4d64e49b1a8e9f4038570e7" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 38007 } } peers { permanent_uuid: "eff72d376a4a460c948075a561a4a611" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41047 } }
I20260808 18:23:29.506793 3094 raft_consensus.cc:374] T 6fce04ac2f644286b90a04ece609f93a P eff72d376a4a460c948075a561a4a611 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:23:29.506850 3094 raft_consensus.cc:729] T 6fce04ac2f644286b90a04ece609f93a P eff72d376a4a460c948075a561a4a611 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eff72d376a4a460c948075a561a4a611, State: Initialized, Role: FOLLOWER
I20260808 18:23:29.506810 3093 raft_consensus.cc:348] T 6fce04ac2f644286b90a04ece609f93a P d24fb014a4d64e49b1a8e9f4038570e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e431ba7482f2472eb0272babad420a55" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 36443 } } peers { permanent_uuid: "d24fb014a4d64e49b1a8e9f4038570e7" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 38007 } } peers { permanent_uuid: "eff72d376a4a460c948075a561a4a611" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41047 } }
I20260808 18:23:29.507283 3093 raft_consensus.cc:374] T 6fce04ac2f644286b90a04ece609f93a P d24fb014a4d64e49b1a8e9f4038570e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:23:29.507359 3093 raft_consensus.cc:729] T 6fce04ac2f644286b90a04ece609f93a P d24fb014a4d64e49b1a8e9f4038570e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d24fb014a4d64e49b1a8e9f4038570e7, State: Initialized, Role: FOLLOWER
I20260808 18:23:29.507376 3094 consensus_queue.cc:260] T 6fce04ac2f644286b90a04ece609f93a P eff72d376a4a460c948075a561a4a611 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e431ba7482f2472eb0272babad420a55" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 36443 } } peers { permanent_uuid: "d24fb014a4d64e49b1a8e9f4038570e7" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 38007 } } peers { permanent_uuid: "eff72d376a4a460c948075a561a4a611" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41047 } }
I20260808 18:23:29.507545 3096 raft_consensus.cc:348] T 6fce04ac2f644286b90a04ece609f93a P e431ba7482f2472eb0272babad420a55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e431ba7482f2472eb0272babad420a55" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 36443 } } peers { permanent_uuid: "d24fb014a4d64e49b1a8e9f4038570e7" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 38007 } } peers { permanent_uuid: "eff72d376a4a460c948075a561a4a611" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41047 } }
I20260808 18:23:29.507827 3096 raft_consensus.cc:374] T 6fce04ac2f644286b90a04ece609f93a P e431ba7482f2472eb0272babad420a55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:23:29.507870 3096 raft_consensus.cc:729] T 6fce04ac2f644286b90a04ece609f93a P e431ba7482f2472eb0272babad420a55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e431ba7482f2472eb0272babad420a55, State: Initialized, Role: FOLLOWER
I20260808 18:23:29.507800 3093 consensus_queue.cc:260] T 6fce04ac2f644286b90a04ece609f93a P d24fb014a4d64e49b1a8e9f4038570e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e431ba7482f2472eb0272babad420a55" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 36443 } } peers { permanent_uuid: "d24fb014a4d64e49b1a8e9f4038570e7" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 38007 } } peers { permanent_uuid: "eff72d376a4a460c948075a561a4a611" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41047 } }
I20260808 18:23:29.508320 3096 consensus_queue.cc:260] T 6fce04ac2f644286b90a04ece609f93a P e431ba7482f2472eb0272babad420a55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e431ba7482f2472eb0272babad420a55" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 36443 } } peers { permanent_uuid: "d24fb014a4d64e49b1a8e9f4038570e7" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 38007 } } peers { permanent_uuid: "eff72d376a4a460c948075a561a4a611" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41047 } }
I20260808 18:23:29.508473 3094 ts_tablet_manager.cc:1428] T 6fce04ac2f644286b90a04ece609f93a P eff72d376a4a460c948075a561a4a611: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260808 18:23:29.508838 2773 heartbeater.cc:503] Master 127.23.72.190:43027 was elected leader, sending a full tablet report...
I20260808 18:23:29.509236 3096 ts_tablet_manager.cc:1428] T 6fce04ac2f644286b90a04ece609f93a P e431ba7482f2472eb0272babad420a55: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 18:23:29.509590 3074 heartbeater.cc:503] Master 127.23.72.190:43027 was elected leader, sending a full tablet report...
I20260808 18:23:29.509913 3093 ts_tablet_manager.cc:1428] T 6fce04ac2f644286b90a04ece609f93a P d24fb014a4d64e49b1a8e9f4038570e7: Time spent starting tablet: real 0.007s user 0.006s sys 0.001s
I20260808 18:23:29.509948 2924 heartbeater.cc:503] Master 127.23.72.190:43027 was elected leader, sending a full tablet report...
I20260808 18:23:29.627928 3100 raft_consensus.cc:482] T 6fce04ac2f644286b90a04ece609f93a P eff72d376a4a460c948075a561a4a611 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:23:29.628201 3100 raft_consensus.cc:504] T 6fce04ac2f644286b90a04ece609f93a P eff72d376a4a460c948075a561a4a611 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e431ba7482f2472eb0272babad420a55" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 36443 } } peers { permanent_uuid: "d24fb014a4d64e49b1a8e9f4038570e7" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 38007 } } peers { permanent_uuid: "eff72d376a4a460c948075a561a4a611" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41047 } }
I20260808 18:23:29.629568 3100 leader_election.cc:288] T 6fce04ac2f644286b90a04ece609f93a P eff72d376a4a460c948075a561a4a611 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e431ba7482f2472eb0272babad420a55 (127.23.72.131:36443), d24fb014a4d64e49b1a8e9f4038570e7 (127.23.72.130:38007)
I20260808 18:23:29.634582 3089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:29.629909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:46119 (local address 127.23.72.131:36443)
0808 18:23:29.630049 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:29.630054 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:29.630254 (+ 200us) server_negotiation.cc:407] Connection header received
0808 18:23:29.630413 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:29.630415 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:29.630460 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:29.630527 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:23:29.630795 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:29.631598 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:29.633231 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:29.633842 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:29.633968 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:29.634019 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:29.634380 (+ 361us) server_negotiation.cc:299] Negotiation successful
0808 18:23:29.634451 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 18:23:29.634581 3092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:29.629967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:55967 (local address 127.23.72.130:38007)
0808 18:23:29.630103 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:29.630110 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:29.630289 (+ 179us) server_negotiation.cc:407] Connection header received
0808 18:23:29.630647 (+ 358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:29.630649 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:29.630691 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:29.630783 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:23:29.631053 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:29.631784 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:29.633231 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:29.633842 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:29.633968 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:29.634019 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:29.634387 (+ 368us) server_negotiation.cc:299] Negotiation successful
0808 18:23:29.634452 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 18:23:29.634663 3103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:29.629806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.131:36443 (local address 127.23.72.129:46119)
0808 18:23:29.630158 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:29.630171 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:29.630294 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:29.630557 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:29.630560 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:29.630576 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:23:29.630673 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:29.630678 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:29.631708 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:29.631711 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:29.633086 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:29.633097 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:29.633978 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:29.633982 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:29.634177 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:29.634264 (+ 87us) client_negotiation.cc:769] Sending connection context
0808 18:23:29.634352 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 18:23:29.634444 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"mutex_wait_us":144,"spinlock_wait_cycles":10752,"thread_start_us":116,"threads_started":1}
I20260808 18:23:29.634964 3104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:29.629791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.130:38007 (local address 127.23.72.129:55967)
0808 18:23:29.630143 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:29.630157 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:23:29.630295 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:23:29.630817 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:23:29.630819 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:23:29.630833 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:23:29.630943 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:29.630947 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:29.631910 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:29.631913 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:29.633086 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:23:29.633097 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:23:29.633978 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:23:29.633982 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:23:29.634216 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:29.634268 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 18:23:29.634344 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 18:23:29.634448 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"mutex_wait_us":108,"spinlock_wait_cycles":22656,"thread_start_us":101,"threads_started":1}
I20260808 18:23:29.635537 3024 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fce04ac2f644286b90a04ece609f93a" candidate_uuid: "eff72d376a4a460c948075a561a4a611" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e431ba7482f2472eb0272babad420a55" is_pre_election: true
I20260808 18:23:29.635656 2869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fce04ac2f644286b90a04ece609f93a" candidate_uuid: "eff72d376a4a460c948075a561a4a611" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d24fb014a4d64e49b1a8e9f4038570e7" is_pre_election: true
I20260808 18:23:29.635933 3024 raft_consensus.cc:2454] T 6fce04ac2f644286b90a04ece609f93a P e431ba7482f2472eb0272babad420a55 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eff72d376a4a460c948075a561a4a611 in term 0.
I20260808 18:23:29.636045 2869 raft_consensus.cc:2454] T 6fce04ac2f644286b90a04ece609f93a P d24fb014a4d64e49b1a8e9f4038570e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eff72d376a4a460c948075a561a4a611 in term 0.
I20260808 18:23:29.636441 2651 leader_election.cc:302] T 6fce04ac2f644286b90a04ece609f93a P eff72d376a4a460c948075a561a4a611 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e431ba7482f2472eb0272babad420a55, eff72d376a4a460c948075a561a4a611; no voters:
I20260808 18:23:29.636782 3100 raft_consensus.cc:2781] T 6fce04ac2f644286b90a04ece609f93a P eff72d376a4a460c948075a561a4a611 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:23:29.636898 3100 raft_consensus.cc:482] T 6fce04ac2f644286b90a04ece609f93a P eff72d376a4a460c948075a561a4a611 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:23:29.636960 3100 raft_consensus.cc:3037] T 6fce04ac2f644286b90a04ece609f93a P eff72d376a4a460c948075a561a4a611 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:23:29.638015 3100 raft_consensus.cc:504] T 6fce04ac2f644286b90a04ece609f93a P eff72d376a4a460c948075a561a4a611 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e431ba7482f2472eb0272babad420a55" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 36443 } } peers { permanent_uuid: "d24fb014a4d64e49b1a8e9f4038570e7" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 38007 } } peers { permanent_uuid: "eff72d376a4a460c948075a561a4a611" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41047 } }
I20260808 18:23:29.638386 3100 leader_election.cc:288] T 6fce04ac2f644286b90a04ece609f93a P eff72d376a4a460c948075a561a4a611 [CANDIDATE]: Term 1 election: Requested vote from peers e431ba7482f2472eb0272babad420a55 (127.23.72.131:36443), d24fb014a4d64e49b1a8e9f4038570e7 (127.23.72.130:38007)
I20260808 18:23:29.638710 3024 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fce04ac2f644286b90a04ece609f93a" candidate_uuid: "eff72d376a4a460c948075a561a4a611" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e431ba7482f2472eb0272babad420a55"
I20260808 18:23:29.638798 2869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fce04ac2f644286b90a04ece609f93a" candidate_uuid: "eff72d376a4a460c948075a561a4a611" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d24fb014a4d64e49b1a8e9f4038570e7"
I20260808 18:23:29.638837 3024 raft_consensus.cc:3037] T 6fce04ac2f644286b90a04ece609f93a P e431ba7482f2472eb0272babad420a55 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:23:29.638974 2869 raft_consensus.cc:3037] T 6fce04ac2f644286b90a04ece609f93a P d24fb014a4d64e49b1a8e9f4038570e7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:23:29.640033 3024 raft_consensus.cc:2454] T 6fce04ac2f644286b90a04ece609f93a P e431ba7482f2472eb0272babad420a55 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eff72d376a4a460c948075a561a4a611 in term 1.
I20260808 18:23:29.640179 2869 raft_consensus.cc:2454] T 6fce04ac2f644286b90a04ece609f93a P d24fb014a4d64e49b1a8e9f4038570e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eff72d376a4a460c948075a561a4a611 in term 1.
I20260808 18:23:29.640329 2651 leader_election.cc:302] T 6fce04ac2f644286b90a04ece609f93a P eff72d376a4a460c948075a561a4a611 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e431ba7482f2472eb0272babad420a55, eff72d376a4a460c948075a561a4a611; no voters:
I20260808 18:23:29.640578 3100 raft_consensus.cc:2781] T 6fce04ac2f644286b90a04ece609f93a P eff72d376a4a460c948075a561a4a611 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:23:29.640813 3100 raft_consensus.cc:686] T 6fce04ac2f644286b90a04ece609f93a P eff72d376a4a460c948075a561a4a611 [term 1 LEADER]: Becoming Leader. State: Replica: eff72d376a4a460c948075a561a4a611, State: Running, Role: LEADER
I20260808 18:23:29.641130 3100 consensus_queue.cc:237] T 6fce04ac2f644286b90a04ece609f93a P eff72d376a4a460c948075a561a4a611 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e431ba7482f2472eb0272babad420a55" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 36443 } } peers { permanent_uuid: "d24fb014a4d64e49b1a8e9f4038570e7" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 38007 } } peers { permanent_uuid: "eff72d376a4a460c948075a561a4a611" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41047 } }
I20260808 18:23:29.644356 2566 catalog_manager.cc:5495] T 6fce04ac2f644286b90a04ece609f93a P eff72d376a4a460c948075a561a4a611 reported cstate change: term changed from 0 to 1, leader changed from <none> to eff72d376a4a460c948075a561a4a611 (127.23.72.129). New cstate: current_term: 1 leader_uuid: "eff72d376a4a460c948075a561a4a611" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e431ba7482f2472eb0272babad420a55" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 36443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d24fb014a4d64e49b1a8e9f4038570e7" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 38007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eff72d376a4a460c948075a561a4a611" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41047 } health_report { overall_health: HEALTHY } } }
I20260808 18:23:29.673341 3091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:29.669744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49720 (local address 127.23.72.129:41047)
0808 18:23:29.669901 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 18:23:29.669905 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:23:29.669918 (+ 13us) server_negotiation.cc:407] Connection header received
0808 18:23:29.669987 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:23:29.669989 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:23:29.670041 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:23:29.670127 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:23:29.670347 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:29.670841 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:29.671875 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:23:29.672124 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:23:29.672206 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:23:29.672471 (+ 265us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:23:29.672903 (+ 432us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:23:29.672995 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 18:23:29.673103 (+ 108us) server_negotiation.cc:299] Negotiation successful
0808 18:23:29.673151 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 18:23:29.678978 2869 raft_consensus.cc:1261] T 6fce04ac2f644286b90a04ece609f93a P d24fb014a4d64e49b1a8e9f4038570e7 [term 1 FOLLOWER]: Refusing update from remote peer eff72d376a4a460c948075a561a4a611: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:23:29.678984 3024 raft_consensus.cc:1261] T 6fce04ac2f644286b90a04ece609f93a P e431ba7482f2472eb0272babad420a55 [term 1 FOLLOWER]: Refusing update from remote peer eff72d376a4a460c948075a561a4a611: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:23:29.679721 3105 consensus_queue.cc:1035] T 6fce04ac2f644286b90a04ece609f93a P eff72d376a4a460c948075a561a4a611 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d24fb014a4d64e49b1a8e9f4038570e7" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:23:29.679927 3100 consensus_queue.cc:1035] T 6fce04ac2f644286b90a04ece609f93a P eff72d376a4a460c948075a561a4a611 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e431ba7482f2472eb0272babad420a55" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 36443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 18:23:29.684082 3075 tablet.cc:2369] T 6fce04ac2f644286b90a04ece609f93a P e431ba7482f2472eb0272babad420a55: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:23:29.689026 3110 mvcc.cc:205] Tried to move back new op lower bound from 7316330126037798912 to 7316330125892468736. Current Snapshot: MvccSnapshot[applied={T|T < 7316330126037798912}]
I20260808 18:23:29.696790 2566 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58320:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:23:29.696975 2566 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58320:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:23:29.699512 2566 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e317d7552be947ecaab3bcc854b04f59: Sending DeleteTablet for 3 replicas of tablet 6fce04ac2f644286b90a04ece609f93a
I20260808 18:23:29.700374 2847 tablet_service.cc:1514] Processing DeleteTablet for tablet 6fce04ac2f644286b90a04ece609f93a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:23:29 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:50270
I20260808 18:23:29.700429 3001 tablet_service.cc:1514] Processing DeleteTablet for tablet 6fce04ac2f644286b90a04ece609f93a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:23:29 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:55486
I20260808 18:23:29.700902 3119 tablet_replica.cc:330] T 6fce04ac2f644286b90a04ece609f93a P d24fb014a4d64e49b1a8e9f4038570e7: stopping tablet replica
I20260808 18:23:29.700888 2703 tablet_service.cc:1514] Processing DeleteTablet for tablet 6fce04ac2f644286b90a04ece609f93a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:23:29 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:49710
I20260808 18:23:29.701242 3119 raft_consensus.cc:2229] T 6fce04ac2f644286b90a04ece609f93a P d24fb014a4d64e49b1a8e9f4038570e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:23:29.701390 3120 tablet_replica.cc:330] T 6fce04ac2f644286b90a04ece609f93a P eff72d376a4a460c948075a561a4a611: stopping tablet replica
I20260808 18:23:29.701566 3119 raft_consensus.cc:2258] T 6fce04ac2f644286b90a04ece609f93a P d24fb014a4d64e49b1a8e9f4038570e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:23:29.701912 3120 raft_consensus.cc:2229] T 6fce04ac2f644286b90a04ece609f93a P eff72d376a4a460c948075a561a4a611 [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:23:29.702206 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 2632
I20260808 18:23:29.703416 3119 ts_tablet_manager.cc:1905] T 6fce04ac2f644286b90a04ece609f93a P d24fb014a4d64e49b1a8e9f4038570e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:23:29.705797 3119 ts_tablet_manager.cc:1918] T 6fce04ac2f644286b90a04ece609f93a P d24fb014a4d64e49b1a8e9f4038570e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:23:29.705893 3119 log.cc:1196] T 6fce04ac2f644286b90a04ece609f93a P d24fb014a4d64e49b1a8e9f4038570e7: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-1/wal/wals/6fce04ac2f644286b90a04ece609f93a
I20260808 18:23:29.706198 3119 ts_tablet_manager.cc:1939] T 6fce04ac2f644286b90a04ece609f93a P d24fb014a4d64e49b1a8e9f4038570e7: Deleting consensus metadata
I20260808 18:23:29.706547 3118 tablet_replica.cc:330] T 6fce04ac2f644286b90a04ece609f93a P e431ba7482f2472eb0272babad420a55: stopping tablet replica
I20260808 18:23:29.706826 3118 raft_consensus.cc:2229] T 6fce04ac2f644286b90a04ece609f93a P e431ba7482f2472eb0272babad420a55 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:23:29.707129 3118 raft_consensus.cc:2258] T 6fce04ac2f644286b90a04ece609f93a P e431ba7482f2472eb0272babad420a55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:23:29.708724 3118 ts_tablet_manager.cc:1905] T 6fce04ac2f644286b90a04ece609f93a P e431ba7482f2472eb0272babad420a55: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:23:29.712097 3118 ts_tablet_manager.cc:1918] T 6fce04ac2f644286b90a04ece609f93a P e431ba7482f2472eb0272babad420a55: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:23:29.712201 3118 log.cc:1196] T 6fce04ac2f644286b90a04ece609f93a P e431ba7482f2472eb0272babad420a55: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213345431840-23842-0/minicluster-data/ts-2/wal/wals/6fce04ac2f644286b90a04ece609f93a
I20260808 18:23:29.712574 3118 ts_tablet_manager.cc:1939] T 6fce04ac2f644286b90a04ece609f93a P e431ba7482f2472eb0272babad420a55: Deleting consensus metadata
I20260808 18:23:29.713809 2550 catalog_manager.cc:4841] TS e431ba7482f2472eb0272babad420a55 (127.23.72.131:36443): tablet 6fce04ac2f644286b90a04ece609f93a (table test-table [id=bad1a19658974b7d84cffeb8ddefe425]) successfully deleted
I20260808 18:23:29.714383 2551 catalog_manager.cc:4841] TS d24fb014a4d64e49b1a8e9f4038570e7 (127.23.72.130:38007): tablet 6fce04ac2f644286b90a04ece609f93a (table test-table [id=bad1a19658974b7d84cffeb8ddefe425]) successfully deleted
W20260808 18:23:29.714934 2550 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.72.129:41047 (error 108)
I20260808 18:23:29.715817 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 2779
I20260808 18:23:29.716041 3090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:23:29.715714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:41047 (local address 127.0.0.1:49734)
0808 18:23:29.715853 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 18:23:29.715936 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.129:41047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 18:23:29.717684 2550 catalog_manager.cc:4584] TS eff72d376a4a460c948075a561a4a611 (127.23.72.129:41047): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6fce04ac2f644286b90a04ece609f93a: Network error: Client connection negotiation failed: client connection to 127.23.72.129:41047: connect: Connection refused (error 111)
I20260808 18:23:29.723294 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 2929
I20260808 18:23:29.730104 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 2532
2026-08-08T18:23:29Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2428 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:23:29 dist-test-slave-5k54 krb5kdc[3124](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:23:29 dist-test-slave-5k54 krb5kdc[3124](info): set up 2 sockets
Aug 08 18:23:29 dist-test-slave-5k54 krb5kdc[3124](info): commencing operation
krb5kdc: starting...
W20260808 18:23:30.809672 23842 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Aug 08 18:23:30 dist-test-slave-5k54 krb5kdc[3124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:23:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:23:30Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.23.72.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.23.72.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.23.72.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/krb5kdc/rangeradmin_127.23.72.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.23.72.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/krb5kdc/rangeradmin_127.23.72.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.23.72.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.23.72.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.23.72.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/krb5kdc/rangerlookup_127.23.72.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.23.72.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/krb5kdc/rangerlookup_127.23.72.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.23.72.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.23.72.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.23.72.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/krb5kdc/HTTP_127.23.72.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.23.72.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/krb5kdc/HTTP_127.23.72.148@KRBTEST.COM.keytab.
I20260808 18:23:30.990327 23842 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskVyYox2/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/postgres -l logfile start
2026-08-08 18:23:33.571 UTC [3154] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 18:23:33.571 UTC [3154] LOG: listening on IPv4 address "127.23.72.148", port 60515
2026-08-08 18:23:33.575 UTC [3154] LOG: listening on Unix socket "/tmp/.s.PGSQL.60515"
2026-08-08 18:23:33.598 UTC [3157] LOG: database system was shut down at 2026-08-08 18:23:32 UTC
2026-08-08 18:23:33.602 UTC [3154] LOG: database system is ready to accept connections
I20260808 18:23:34.684556 23842 mini_postgres.cc:96] Postgres bound to 60515
2026-08-08 18:23:34.692 UTC [3166] FATAL: database "slave" does not exist
127.23.72.148:60515 - accepting connections
I20260808 18:23:34.693621 23842 mini_ranger.cc:161] Starting Ranger...
I20260808 18:23:34.711565 23842 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 18:23:35.194316 23842 mini_ranger.cc:87] Created ranger Postgres database
I20260808 18:23:35.194420 23842 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ranger-admin
2026-08-08 18:23:35,509 [I] DB FLAVOR :POSTGRES
2026-08-08 18:23:35,510 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 18:23:35,510 [I] Checking connection..
2026-08-08 18:23:35,510 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.72.148:60515/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 18:23:35,780 [I] Checking connection passed.
2026-08-08 18:23:35,780 [I] --------- Verifying version history table ---------
2026-08-08 18:23:35,780 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.72.148:60515/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 18:23:36,039 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 18:23:36,039 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.72.148:60515/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 18:23:36,297 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 18:23:36,297 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 18:23:36,297 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.72.148:60515/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 18:23:36.542 UTC [3233] WARNING: there is no transaction in progress
2026-08-08 18:23:36,549 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 18:23:36,549 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.72.148:60515/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 18:23:36,795 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 18:23:36,795 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 18:23:36,796 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.72.148:60515/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 18:23:37,058 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.72.148:60515/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 18:23:37,306 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.72.148:60515/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-5k54.c.gcp-upstream.internal','N') ;"
2026-08-08 18:23:37,542 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.72.148:60515/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 18:23:37,804 [I] Table x_portal_user does not exist in database ranger
2026-08-08 18:23:37,805 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.72.148:60515/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 18:23:38,052 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 18:23:38,052 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.72.148:60515/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 18:23:38,296 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.72.148:60515/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 18:23:38,542 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 18:23:38,543 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.72.148:60515/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 18:23:39.534 UTC [3354] WARNING: there is no transaction in progress
2026-08-08 18:23:39.552 UTC [3354] WARNING: there is no transaction in progress
2026-08-08 18:23:39.569 UTC [3354] WARNING: there is no transaction in progress
2026-08-08 18:23:39.585 UTC [3354] WARNING: there is no transaction in progress
2026-08-08 18:23:39.602 UTC [3354] WARNING: there is no transaction in progress
2026-08-08 18:23:39.621 UTC [3354] WARNING: there is no transaction in progress
2026-08-08 18:23:39.677 UTC [3354] WARNING: there is no transaction in progress
2026-08-08 18:23:39.684 UTC [3354] WARNING: there is no transaction in progress
2026-08-08 18:23:39.698 UTC [3354] WARNING: there is no transaction in progress
2026-08-08 18:23:39.707 UTC [3354] WARNING: there is no transaction in progress
2026-08-08 18:23:39.717 UTC [3354] WARNING: there is no transaction in progress
2026-08-08 18:23:39.730 UTC [3354] WARNING: there is no transaction in progress
2026-08-08 18:23:39.740 UTC [3354] WARNING: there is no transaction in progress
2026-08-08 18:23:39.746 UTC [3354] WARNING: there is no transaction in progress
2026-08-08 18:23:39.753 UTC [3354] WARNING: there is no transaction in progress
2026-08-08 18:23:39.760 UTC [3354] WARNING: there is no transaction in progress
2026-08-08 18:23:40,398 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 18:23:40,398 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.72.148:60515/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 18:23:40,655 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 18:23:40,656 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.72.148:60515/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 18:23:40,921 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 18:23:40,922 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.72.148:60515/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 18:23:41,166 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 18:23:41,167 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.72.148:60515/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 18:23:41,409 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.72.148:60515/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 18:23:41,676 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.72.148:60515/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-5k54.c.gcp-upstream.internal';"
2026-08-08 18:23:41,926 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 18:23:41,926 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.72.148:60515/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 18:23:42,172 [I] DB_PATCHES have already been applied
I20260808 18:23:42.177888 23842 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ranger-admin:/tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskVyYox2/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 18:23:42.177959 23842 mini_ranger.cc:193] Using host: 127.23.72.148
I20260808 18:23:42.182435 23842 mini_ranger.cc:227] Ranger admin URL: http://127.23.72.148:33689
Aug 08, 2026 6:23:42 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 6:23:42 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 6:23:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 6:23:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 6:23:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 6:23:43 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 6:23:43 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 18:23:43.352881 23842 mini_ranger.cc:160] Time spent starting Ranger: real 8.659s user 0.004s sys 0.009s
Aug 08, 2026 6:24:03 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 6:24:03 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 08, 2026 6:24:03 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 6:24:04 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 6:24:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 6:24:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 6:24:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 6:24:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 6:24:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 6:24:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 6:24:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 6:24:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 6:24:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 6:24:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 6:24:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 6:24:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 6:24:04 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 6:24:04 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 18:24:05.561793 23842 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.23.72.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.23.72.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.23.72.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/krb5kdc/rangerkms_127.23.72.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.23.72.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/krb5kdc/rangerkms_127.23.72.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.23.72.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.23.72.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.23.72.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/krb5kdc/HTTP_127.23.72.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.23.72.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/krb5kdc/HTTP_127.23.72.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 18:24:05 dist-test-slave-5k54 krb5kdc[3124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213445, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 18:24:05.654419 23842 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 18:24:05.667124 23842 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 18:24:06.586099 23842 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 18:24:06.586205 23842 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ranger-kms
I20260808 18:24:06.586318 23842 mini_ranger_kms.cc:227] Using postgres at 127.23.72.148:60515
2026-08-08 18:24:07,042 [I] DB FLAVOR :POSTGRES
2026-08-08 18:24:07,042 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 18:24:07,042 [I] Checking connection
2026-08-08 18:24:07,042 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.72.148:60515/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 18:24:07,353 [I] connection success
2026-08-08 18:24:07,353 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 18:24:07,353 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 18:24:07,353 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.72.148:60515/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 18:24:07,633 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 18:24:07,633 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 18:24:07,633 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 18:24:07,633 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.72.148:60515/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 18:24:07,908 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 18:24:08.384680 23842 mini_ranger_kms.cc:310] Created kms service
I20260808 18:24:08.574505 23842 mini_ranger_kms.cc:326] Created kudu user
I20260808 18:24:08.621057 23842 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 18:24:08 dist-test-slave-5k54 krb5kdc[3124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213445, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.23.72.148@KRBTEST.COM
I20260808 18:24:09.010675 23842 mini_ranger_kms.cc:384] Added ranger policy
I20260808 18:24:09.010936 23842 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ranger-kms:/tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 18:24:09.010975 23842 mini_ranger_kms.cc:240] Using host: 127.23.72.148
I20260808 18:24:09.016336 23842 mini_ranger_kms.cc:276] Ranger KMS PID: 3548
I20260808 18:24:09.016462 23842 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.23.72.148:51389
Aug 08, 2026 6:24:10 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 6:24:10 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 6:24:10 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 6:24:10 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 6:24:10 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 6:24:10 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260808 18:24:11.418045 23842 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.764s user 0.006s sys 0.412s
I20260808 18:24:11.418195 23842 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 18:24:11.418321 23842 mini_ranger_kms.cc:401] 127.23.72.148:51389/kms/v1/keys
Aug 08 18:24:11 dist-test-slave-5k54 krb5kdc[3124](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786213451, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.23.72.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 08 18:24:12 dist-test-slave-5k54 krb5kdc[3124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213451, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.23.72.148@KRBTEST.COM for HTTP/127.23.72.148@KRBTEST.COM
Aug 08 18:24:12 dist-test-slave-5k54 krb5kdc[3124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213451, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.23.72.148@KRBTEST.COM for HTTP/127.23.72.148@KRBTEST.COM
Aug 08 18:24:15 dist-test-slave-5k54 krb5kdc[3124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213455, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.23.72.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
I20260808 18:24:15.090845 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:40295
--webserver_interface=127.23.72.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:43773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:40295
--ranger_config_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.72.148:51389/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:24:15.236363 3604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:24:15.236680 3604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:24:15.236760 3604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:24:15.240825 3604 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:24:15.240909 3604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:24:15.240947 3604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:24:15.241005 3604 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:24:15.241042 3604 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:24:15.246778 3604 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:43773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/master-0/wal
--ranger_kms_url=127.23.72.148:51389/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:40295
--ranger_config_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:40295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.72.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3604
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:24:15.247984 3604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:24:15.249038 3604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:24:15.260601 3610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:24:15.260664 3613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:24:15.260601 3611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:24:15.261003 3604 server_base.cc:1034] running on GCE node
I20260808 18:24:15.261564 3604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:24:15.262746 3604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:24:15.263940 3604 hybrid_clock.cc:648] HybridClock initialized: now 1786213455263923 us; error 36 us; skew 500 ppm
Aug 08 18:24:15 dist-test-slave-5k54 krb5kdc[3124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213455, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:24:15.267395 3604 init.cc:377] Logged in from keytab as kudu/127.23.72.190@KRBTEST.COM (short username kudu)
I20260808 18:24:15.268529 3604 webserver.cc:460] Webserver started at http://127.23.72.190:40343/ using document root <none> and password file <none>
I20260808 18:24:15.269085 3604 fs_manager.cc:359] Metadata directory not provided
I20260808 18:24:15.269169 3604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:24:15.269387 3604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 18:24:15 dist-test-slave-5k54 krb5kdc[3124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213455, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for HTTP/127.23.72.148@KRBTEST.COM
I20260808 18:24:15.321962 3604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "59b31f9e14ec40508b714b59729ce179"
format_stamp: "Formatted at 2026-08-08 18:24:15 on dist-test-slave-5k54"
server_key: "a02064e428d35ae492ce28fac8b85e5f"
server_key_iv: "f7b6960f6ccbd917ea349d5a7e6c2c44"
server_key_version: "kuduclusterkey@0"
I20260808 18:24:15.322654 3604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "59b31f9e14ec40508b714b59729ce179"
format_stamp: "Formatted at 2026-08-08 18:24:15 on dist-test-slave-5k54"
server_key: "a02064e428d35ae492ce28fac8b85e5f"
server_key_iv: "f7b6960f6ccbd917ea349d5a7e6c2c44"
server_key_version: "kuduclusterkey@0"
I20260808 18:24:15.326912 3604 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 18:24:15.341420 3620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:24:15.342905 3604 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 18:24:15.343017 3604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "59b31f9e14ec40508b714b59729ce179"
format_stamp: "Formatted at 2026-08-08 18:24:15 on dist-test-slave-5k54"
server_key: "a02064e428d35ae492ce28fac8b85e5f"
server_key_iv: "f7b6960f6ccbd917ea349d5a7e6c2c44"
server_key_version: "kuduclusterkey@0"
I20260808 18:24:15.343178 3604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:24:15.364856 3604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:24:15.367911 3604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:24:15.368153 3604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:24:15.381423 3604 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.190:40295
I20260808 18:24:15.381487 3677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.190:40295 every 8 connection(s)
I20260808 18:24:15.382525 3604 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:24:15.385813 3678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:24:15.390457 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 3604
I20260808 18:24:15.390602 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:24:15.390908 23842 external_mini_cluster.cc:1428] Setting key 8a0a4ece02f970ceb8e402d0e2927475
I20260808 18:24:15.392761 3678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59b31f9e14ec40508b714b59729ce179: Bootstrap starting.
I20260808 18:24:15.396215 3678 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59b31f9e14ec40508b714b59729ce179: Neither blocks nor log segments found. Creating new log.
I20260808 18:24:15.397363 3678 log.cc:824] T 00000000000000000000000000000000 P 59b31f9e14ec40508b714b59729ce179: Log is configured to *not* fsync() on all Append() calls
I20260808 18:24:15.399611 3678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59b31f9e14ec40508b714b59729ce179: No bootstrap required, opened a new log
Aug 08 18:24:15 dist-test-slave-5k54 krb5kdc[3124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213455, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:24:15.404971 3678 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59b31f9e14ec40508b714b59729ce179 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59b31f9e14ec40508b714b59729ce179" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40295 } }
I20260808 18:24:15.405272 3678 raft_consensus.cc:374] T 00000000000000000000000000000000 P 59b31f9e14ec40508b714b59729ce179 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:24:15.405395 3678 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59b31f9e14ec40508b714b59729ce179 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59b31f9e14ec40508b714b59729ce179, State: Initialized, Role: FOLLOWER
I20260808 18:24:15.405912 3678 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59b31f9e14ec40508b714b59729ce179 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59b31f9e14ec40508b714b59729ce179" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40295 } }
I20260808 18:24:15.406078 3678 raft_consensus.cc:388] T 00000000000000000000000000000000 P 59b31f9e14ec40508b714b59729ce179 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:24:15.406173 3678 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59b31f9e14ec40508b714b59729ce179 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:24:15.406320 3678 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59b31f9e14ec40508b714b59729ce179 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:24:15.407603 3678 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59b31f9e14ec40508b714b59729ce179 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59b31f9e14ec40508b714b59729ce179" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40295 } }
I20260808 18:24:15.407990 3678 leader_election.cc:302] T 00000000000000000000000000000000 P 59b31f9e14ec40508b714b59729ce179 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 59b31f9e14ec40508b714b59729ce179; no voters:
I20260808 18:24:15.408418 3678 leader_election.cc:288] T 00000000000000000000000000000000 P 59b31f9e14ec40508b714b59729ce179 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:24:15.408870 3684 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 59b31f9e14ec40508b714b59729ce179 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:24:15.409179 3684 raft_consensus.cc:686] T 00000000000000000000000000000000 P 59b31f9e14ec40508b714b59729ce179 [term 1 LEADER]: Becoming Leader. State: Replica: 59b31f9e14ec40508b714b59729ce179, State: Running, Role: LEADER
I20260808 18:24:15.409550 3684 consensus_queue.cc:237] T 00000000000000000000000000000000 P 59b31f9e14ec40508b714b59729ce179 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59b31f9e14ec40508b714b59729ce179" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40295 } }
I20260808 18:24:15.409667 3678 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59b31f9e14ec40508b714b59729ce179 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:24:15.409718 3681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:24:15.392430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60262 (local address 127.23.72.190:40295)
0808 18:24:15.392983 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0808 18:24:15.393053 (+ 70us) server_negotiation.cc:399] Waiting for connection header
0808 18:24:15.393096 (+ 43us) server_negotiation.cc:407] Connection header received
0808 18:24:15.394318 (+ 1222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:24:15.394374 (+ 56us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:24:15.394980 (+ 606us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:24:15.395339 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:24:15.396392 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:24:15.398214 (+ 1822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:24:15.399387 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:24:15.400160 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:24:15.400319 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:24:15.403761 (+ 3442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:24:15.403769 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:24:15.403803 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:24:15.403835 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:24:15.406087 (+ 2252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:24:15.406662 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:24:15.406669 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:24:15.406674 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:24:15.406801 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:24:15.407104 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:24:15.407107 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:24:15.407109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:24:15.407759 (+ 650us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:24:15.407975 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0808 18:24:15.408260 (+ 285us) server_negotiation.cc:299] Negotiation successful
0808 18:24:15.408727 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":175,"threads_started":1}
I20260808 18:24:15.412815 3683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59b31f9e14ec40508b714b59729ce179 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "59b31f9e14ec40508b714b59729ce179" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59b31f9e14ec40508b714b59729ce179" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40295 } } }
I20260808 18:24:15.412999 3683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59b31f9e14ec40508b714b59729ce179 [sys.catalog]: This master's current role is: LEADER
I20260808 18:24:15.413255 3683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59b31f9e14ec40508b714b59729ce179 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59b31f9e14ec40508b714b59729ce179. Latest consensus state: current_term: 1 leader_uuid: "59b31f9e14ec40508b714b59729ce179" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59b31f9e14ec40508b714b59729ce179" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40295 } } }
I20260808 18:24:15.413342 3683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59b31f9e14ec40508b714b59729ce179 [sys.catalog]: This master's current role is: LEADER
I20260808 18:24:15.413477 3697 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:24:15.413627 3678 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 18:24:15.423267 3697 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:24:15.437448 3697 catalog_manager.cc:1347] Generated new cluster ID: 52e098f6a60140b6bfb0036d3a8d89be
I20260808 18:24:15.437525 3697 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:24:15.455310 3697 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:24:15.456637 3697 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:24:15.467492 3697 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 59b31f9e14ec40508b714b59729ce179: Generated new TSK 0
I20260808 18:24:15.468266 3697 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.72.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab.
Entry for principal kudu/127.23.72.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab.
I20260808 18:24:16.363514 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.72.148:51389/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.129:0
--local_ip_for_outbound_sockets=127.23.72.129
--webserver_interface=127.23.72.129
--webserver_port=0
--tserver_master_addrs=127.23.72.190:40295
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab
--principal=kudu/127.23.72.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:43773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:24:16.533196 3725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:24:16.533476 3725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:24:16.533591 3725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:24:16.537947 3725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:24:16.538061 3725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:24:16.538166 3725 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.129
I20260808 18:24:16.545148 3725 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:43773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.23.72.148:51389/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab
--principal=kudu/127.23.72.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.72.129
--webserver_port=0
--tserver_master_addrs=127.23.72.190:40295
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.129
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:24:16.546665 3725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:24:16.548076 3725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:24:16.559628 3734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:24:16.560003 3735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:24:16.560218 3737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:24:16.560725 3725 server_base.cc:1034] running on GCE node
I20260808 18:24:16.561189 3725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:24:16.561954 3725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:24:16.563133 3725 hybrid_clock.cc:648] HybridClock initialized: now 1786213456563056 us; error 91 us; skew 500 ppm
Aug 08 18:24:16 dist-test-slave-5k54 krb5kdc[3124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213456, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:24:16.567674 3725 init.cc:377] Logged in from keytab as kudu/127.23.72.129@KRBTEST.COM (short username kudu)
I20260808 18:24:16.569047 3725 webserver.cc:460] Webserver started at http://127.23.72.129:34033/ using document root <none> and password file <none>
I20260808 18:24:16.569940 3725 fs_manager.cc:359] Metadata directory not provided
I20260808 18:24:16.570061 3725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:24:16.570390 3725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 18:24:16 dist-test-slave-5k54 krb5kdc[3124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213456, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for HTTP/127.23.72.148@KRBTEST.COM
I20260808 18:24:16.588385 3725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-0/data/instance:
uuid: "a0b0d83253554fe78a3fec33f2267ceb"
format_stamp: "Formatted at 2026-08-08 18:24:16 on dist-test-slave-5k54"
server_key: "feef82ea87b1bffd32a0b77f8d15e358"
server_key_iv: "c4754f874c842c4b1332e735b81ae834"
server_key_version: "kuduclusterkey@0"
I20260808 18:24:16.589635 3725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance:
uuid: "a0b0d83253554fe78a3fec33f2267ceb"
format_stamp: "Formatted at 2026-08-08 18:24:16 on dist-test-slave-5k54"
server_key: "feef82ea87b1bffd32a0b77f8d15e358"
server_key_iv: "c4754f874c842c4b1332e735b81ae834"
server_key_version: "kuduclusterkey@0"
I20260808 18:24:16.601765 3725 fs_manager.cc:691] Time spent creating directory manager: real 0.012s user 0.001s sys 0.004s
I20260808 18:24:16.623898 3744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:24:16.625801 3725 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.000s
I20260808 18:24:16.625952 3725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-0/wal
uuid: "a0b0d83253554fe78a3fec33f2267ceb"
format_stamp: "Formatted at 2026-08-08 18:24:16 on dist-test-slave-5k54"
server_key: "feef82ea87b1bffd32a0b77f8d15e358"
server_key_iv: "c4754f874c842c4b1332e735b81ae834"
server_key_version: "kuduclusterkey@0"
I20260808 18:24:16.626091 3725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:24:16.658854 3725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:24:16.663313 3725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:24:16.663617 3725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:24:16.664711 3725 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:24:16.666132 3725 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:24:16.666249 3725 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:24:16.666384 3725 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:24:16.666507 3725 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:24:16.695104 3725 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.129:42449
I20260808 18:24:16.696568 3725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
I20260808 18:24:16.698050 3867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.129:42449 every 8 connection(s)
I20260808 18:24:16.699496 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 3725
I20260808 18:24:16.699623 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance
I20260808 18:24:16.699970 23842 external_mini_cluster.cc:1428] Setting key d4c5a8c0ad9b95d7188a9d55a73fc972
Aug 08 18:24:16 dist-test-slave-5k54 krb5kdc[3124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213456, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.72.130@KRBTEST.COM; defaulting to no policy
I20260808 18:24:16.724136 3871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:24:16.703063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:48603 (local address 127.23.72.190:40295)
0808 18:24:16.707191 (+ 4128us) server_negotiation.cc:206] Beginning negotiation
0808 18:24:16.707196 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:24:16.708816 (+ 1620us) server_negotiation.cc:407] Connection header received
0808 18:24:16.710182 (+ 1366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:24:16.710186 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:24:16.710243 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:24:16.710311 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:24:16.712383 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:24:16.713281 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:24:16.715041 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:24:16.715405 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:24:16.715518 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:24:16.719388 (+ 3870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:24:16.719392 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:24:16.719395 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:24:16.719450 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:24:16.721517 (+ 2067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:24:16.722222 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:24:16.722225 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:24:16.722227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:24:16.722321 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:24:16.722752 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:24:16.722756 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:24:16.722758 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:24:16.723001 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:24:16.723077 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 18:24:16.723799 (+ 722us) server_negotiation.cc:299] Negotiation successful
0808 18:24:16.723945 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4036,"thread_start_us":90,"threads_started":1}
I20260808 18:24:16.726155 3870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:24:16.701935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:40295 (local address 127.23.72.129:48603)
0808 18:24:16.708620 (+ 6685us) negotiation.cc:107] Waiting for socket to connect
0808 18:24:16.708689 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0808 18:24:16.709921 (+ 1232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:24:16.710768 (+ 847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:24:16.710779 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:24:16.711332 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:24:16.712201 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:24:16.712223 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:24:16.713454 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:24:16.713457 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:24:16.714905 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:24:16.714915 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:24:16.715732 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:24:16.715736 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:24:16.715991 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:24:16.716826 (+ 835us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:24:16.716847 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:24:16.719173 (+ 2326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:24:16.721692 (+ 2519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:24:16.721698 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:24:16.721715 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:24:16.722100 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:24:16.722438 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:24:16.722441 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:24:16.722443 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:24:16.722626 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:24:16.723160 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:24:16.723166 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:24:16.723510 (+ 344us) client_negotiation.cc:769] Sending connection context
0808 18:24:16.724615 (+ 1105us) client_negotiation.cc:240] Negotiation successful
0808 18:24:16.725001 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6425,"thread_start_us":248,"threads_started":1}
I20260808 18:24:16.728037 3868 heartbeater.cc:347] Connected to a master server at 127.23.72.190:40295
I20260808 18:24:16.728291 3868 heartbeater.cc:464] Registering TS with master...
I20260808 18:24:16.728873 3868 heartbeater.cc:511] Master 127.23.72.190:40295 requested a full tablet report, sending...
I20260808 18:24:16.730690 3639 ts_manager.cc:194] Registered new tserver with Master: a0b0d83253554fe78a3fec33f2267ceb (127.23.72.129:42449)
I20260808 18:24:16.732350 3639 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.129@KRBTEST.COM'} at 127.23.72.129:48603
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab.
Entry for principal kudu/127.23.72.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab.
I20260808 18:24:16.760031 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.72.148:51389/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.130:0
--local_ip_for_outbound_sockets=127.23.72.130
--webserver_interface=127.23.72.130
--webserver_port=0
--tserver_master_addrs=127.23.72.190:40295
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab
--principal=kudu/127.23.72.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:43773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 18:24:16 dist-test-slave-5k54 krb5kdc[3124](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786213456, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 18:24:17.014923 3877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:24:17.015486 3877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:24:17.015628 3877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:24:17.023582 3877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:24:17.023696 3877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:24:17.023819 3877 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.130
I20260808 18:24:17.034126 3877 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:43773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.23.72.148:51389/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab
--principal=kudu/127.23.72.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.72.130
--webserver_port=0
--tserver_master_addrs=127.23.72.190:40295
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.130
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:24:17.036127 3877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:24:17.037592 3877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:24:17.056320 3886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:24:17.056969 3877 server_base.cc:1034] running on GCE node
W20260808 18:24:17.057588 3884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:24:17.058460 3883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:24:17.059093 3877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:24:17.059919 3877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:24:17.061213 3877 hybrid_clock.cc:648] HybridClock initialized: now 1786213457061165 us; error 73 us; skew 500 ppm
Aug 08 18:24:17 dist-test-slave-5k54 krb5kdc[3124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213457, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:24:17.065615 3877 init.cc:377] Logged in from keytab as kudu/127.23.72.130@KRBTEST.COM (short username kudu)
I20260808 18:24:17.066735 3877 webserver.cc:460] Webserver started at http://127.23.72.130:42701/ using document root <none> and password file <none>
I20260808 18:24:17.067400 3877 fs_manager.cc:359] Metadata directory not provided
I20260808 18:24:17.067492 3877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:24:17.067711 3877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 18:24:17 dist-test-slave-5k54 krb5kdc[3124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213457, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.130@KRBTEST.COM for HTTP/127.23.72.148@KRBTEST.COM
I20260808 18:24:17.084053 3877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-1/data/instance:
uuid: "b27e7586d9f04315ae3f607bf1a10343"
format_stamp: "Formatted at 2026-08-08 18:24:17 on dist-test-slave-5k54"
server_key: "e516b5d816b8f7f88c99fe330d5219c9"
server_key_iv: "8e43e3181ce9bda23fa86fbcf2ff9787"
server_key_version: "kuduclusterkey@0"
I20260808 18:24:17.084959 3877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance:
uuid: "b27e7586d9f04315ae3f607bf1a10343"
format_stamp: "Formatted at 2026-08-08 18:24:17 on dist-test-slave-5k54"
server_key: "e516b5d816b8f7f88c99fe330d5219c9"
server_key_iv: "8e43e3181ce9bda23fa86fbcf2ff9787"
server_key_version: "kuduclusterkey@0"
I20260808 18:24:17.091655 3877 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260808 18:24:17.109628 3894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:24:17.111572 3877 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260808 18:24:17.111707 3877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-1/wal
uuid: "b27e7586d9f04315ae3f607bf1a10343"
format_stamp: "Formatted at 2026-08-08 18:24:17 on dist-test-slave-5k54"
server_key: "e516b5d816b8f7f88c99fe330d5219c9"
server_key_iv: "8e43e3181ce9bda23fa86fbcf2ff9787"
server_key_version: "kuduclusterkey@0"
I20260808 18:24:17.111845 3877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:24:17.142135 3877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:24:17.146382 3877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:24:17.146661 3877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:24:17.147668 3877 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:24:17.149029 3877 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:24:17.149133 3877 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:24:17.149235 3877 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:24:17.149322 3877 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:24:17.171237 3877 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.130:37803
I20260808 18:24:17.172631 3877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
I20260808 18:24:17.179015 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 3877
I20260808 18:24:17.179152 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance
I20260808 18:24:17.179504 23842 external_mini_cluster.cc:1428] Setting key cf3c9ff23c92ddd2a6b3d419277833e3
I20260808 18:24:17.185351 4016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.130:37803 every 8 connection(s)
Aug 08 18:24:17 dist-test-slave-5k54 krb5kdc[3124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213457, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.130@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.72.131@KRBTEST.COM; defaulting to no policy
I20260808 18:24:17.206892 3871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:24:17.187816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:60167 (local address 127.23.72.190:40295)
0808 18:24:17.187961 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 18:24:17.187966 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:24:17.189879 (+ 1913us) server_negotiation.cc:407] Connection header received
0808 18:24:17.191357 (+ 1478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:24:17.191361 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:24:17.191414 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:24:17.191499 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:24:17.193480 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:24:17.194337 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:24:17.196110 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:24:17.196474 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:24:17.196547 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:24:17.201330 (+ 4783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:24:17.201340 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:24:17.201342 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:24:17.201367 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:24:17.203698 (+ 2331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:24:17.204512 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:24:17.204518 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:24:17.204522 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:24:17.204625 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:24:17.205216 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:24:17.205221 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:24:17.205226 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:24:17.205530 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:24:17.205652 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 18:24:17.206484 (+ 832us) server_negotiation.cc:299] Negotiation successful
0808 18:24:17.206645 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 18:24:17.208341 4020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:24:17.188326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:40295 (local address 127.23.72.130:60167)
0808 18:24:17.189716 (+ 1390us) negotiation.cc:107] Waiting for socket to connect
0808 18:24:17.189777 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 18:24:17.191101 (+ 1324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:24:17.191913 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:24:17.191933 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:24:17.192427 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:24:17.193295 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:24:17.193317 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:24:17.194501 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:24:17.194504 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:24:17.195970 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:24:17.195981 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:24:17.196743 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:24:17.196746 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:24:17.196993 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:24:17.197830 (+ 837us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:24:17.197850 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:24:17.201120 (+ 3270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:24:17.203877 (+ 2757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:24:17.203886 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:24:17.203915 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:24:17.204366 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:24:17.204788 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:24:17.204791 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:24:17.204793 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:24:17.205029 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:24:17.205749 (+ 720us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:24:17.205756 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:24:17.206162 (+ 406us) client_negotiation.cc:769] Sending connection context
0808 18:24:17.207101 (+ 939us) client_negotiation.cc:240] Negotiation successful
0808 18:24:17.207489 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1143,"thread_start_us":213,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.131@KRBTEST.COM" created.
I20260808 18:24:17.212826 4017 heartbeater.cc:347] Connected to a master server at 127.23.72.190:40295
I20260808 18:24:17.213110 4017 heartbeater.cc:464] Registering TS with master...
I20260808 18:24:17.213686 4017 heartbeater.cc:511] Master 127.23.72.190:40295 requested a full tablet report, sending...
I20260808 18:24:17.214875 3639 ts_manager.cc:194] Registered new tserver with Master: b27e7586d9f04315ae3f607bf1a10343 (127.23.72.130:37803)
I20260808 18:24:17.215665 3639 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.130@KRBTEST.COM'} at 127.23.72.130:60167
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab.
Entry for principal kudu/127.23.72.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab.
I20260808 18:24:17.231699 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.72.148:51389/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.131:0
--local_ip_for_outbound_sockets=127.23.72.131
--webserver_interface=127.23.72.131
--webserver_port=0
--tserver_master_addrs=127.23.72.190:40295
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab
--principal=kudu/127.23.72.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:43773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:24:17.417654 4022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:24:17.418067 4022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:24:17.418198 4022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:24:17.424361 4022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:24:17.424503 4022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:24:17.424631 4022 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.131
I20260808 18:24:17.433250 4022 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:43773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.23.72.148:51389/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab
--principal=kudu/127.23.72.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.72.131
--webserver_port=0
--tserver_master_addrs=127.23.72.190:40295
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.131
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:24:17.435022 4022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:24:17.436563 4022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:24:17.450994 4027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:24:17.451009 4028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:24:17.451385 4030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:24:17.452158 4022 server_base.cc:1034] running on GCE node
I20260808 18:24:17.452800 4022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:24:17.453611 4022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:24:17.454815 4022 hybrid_clock.cc:648] HybridClock initialized: now 1786213457454783 us; error 46 us; skew 500 ppm
Aug 08 18:24:17 dist-test-slave-5k54 krb5kdc[3124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213457, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:24:17.459152 4022 init.cc:377] Logged in from keytab as kudu/127.23.72.131@KRBTEST.COM (short username kudu)
I20260808 18:24:17.460237 4022 webserver.cc:460] Webserver started at http://127.23.72.131:42655/ using document root <none> and password file <none>
I20260808 18:24:17.461062 4022 fs_manager.cc:359] Metadata directory not provided
I20260808 18:24:17.461148 4022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:24:17.461376 4022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 18:24:17 dist-test-slave-5k54 krb5kdc[3124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213457, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.131@KRBTEST.COM for HTTP/127.23.72.148@KRBTEST.COM
I20260808 18:24:17.487293 4022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-2/data/instance:
uuid: "6fba1af734cc40849b5e3bbf21a01425"
format_stamp: "Formatted at 2026-08-08 18:24:17 on dist-test-slave-5k54"
server_key: "67b0259c0afb59f73bade7948b91ded6"
server_key_iv: "f0f41af7c74c74bef38524bee8ec4339"
server_key_version: "kuduclusterkey@0"
I20260808 18:24:17.488258 4022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance:
uuid: "6fba1af734cc40849b5e3bbf21a01425"
format_stamp: "Formatted at 2026-08-08 18:24:17 on dist-test-slave-5k54"
server_key: "67b0259c0afb59f73bade7948b91ded6"
server_key_iv: "f0f41af7c74c74bef38524bee8ec4339"
server_key_version: "kuduclusterkey@0"
I20260808 18:24:17.494307 4022 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260808 18:24:17.531983 4037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:24:17.534015 4022 fs_manager.cc:725] Time spent opening block manager: real 0.019s user 0.003s sys 0.000s
I20260808 18:24:17.534163 4022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-2/wal
uuid: "6fba1af734cc40849b5e3bbf21a01425"
format_stamp: "Formatted at 2026-08-08 18:24:17 on dist-test-slave-5k54"
server_key: "67b0259c0afb59f73bade7948b91ded6"
server_key_iv: "f0f41af7c74c74bef38524bee8ec4339"
server_key_version: "kuduclusterkey@0"
I20260808 18:24:17.534327 4022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 08 18:24:17 dist-test-slave-5k54 krb5kdc[3124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213456, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for HTTP/127.23.72.148@KRBTEST.COM
I20260808 18:24:17.553138 4022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:24:17.556519 4022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:24:17.556757 4022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:24:17.557588 4022 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:24:17.559032 4022 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:24:17.559149 4022 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:24:17.559255 4022 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:24:17.559340 4022 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:24:17.593500 4022 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.131:42699
I20260808 18:24:17.594990 4022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
I20260808 18:24:17.604275 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 4022
I20260808 18:24:17.604455 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance
I20260808 18:24:17.604799 23842 external_mini_cluster.cc:1428] Setting key 4d9a0fb620d173dd1187cdbea1bbf4fc
I20260808 18:24:17.602007 4156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.131:42699 every 8 connection(s)
Aug 08 18:24:17 dist-test-slave-5k54 krb5kdc[3124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213457, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.131@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:24:17.627499 3871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:24:17.605823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:39445 (local address 127.23.72.190:40295)
0808 18:24:17.606082 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0808 18:24:17.606086 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:24:17.611964 (+ 5878us) server_negotiation.cc:407] Connection header received
0808 18:24:17.613506 (+ 1542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:24:17.613509 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:24:17.613561 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:24:17.613612 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:24:17.615600 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:24:17.616566 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:24:17.619057 (+ 2491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:24:17.619376 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:24:17.619586 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:24:17.623181 (+ 3595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:24:17.623185 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:24:17.623188 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:24:17.623217 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:24:17.625073 (+ 1856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:24:17.625700 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:24:17.625704 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:24:17.625706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:24:17.625790 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:24:17.626174 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:24:17.626178 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:24:17.626179 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:24:17.626386 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:24:17.626468 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 18:24:17.627156 (+ 688us) server_negotiation.cc:299] Negotiation successful
0808 18:24:17.627280 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178}
I20260808 18:24:17.629375 4161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:24:17.605166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:40295 (local address 127.23.72.131:39445)
0808 18:24:17.611799 (+ 6633us) negotiation.cc:107] Waiting for socket to connect
0808 18:24:17.611862 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 18:24:17.613250 (+ 1388us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:24:17.614022 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:24:17.614031 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:24:17.614571 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:24:17.615407 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:24:17.615450 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:24:17.616735 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:24:17.616739 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:24:17.618924 (+ 2185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:24:17.618935 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:24:17.619829 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:24:17.619833 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:24:17.620084 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:24:17.620819 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:24:17.620840 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:24:17.622973 (+ 2133us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:24:17.625207 (+ 2234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:24:17.625213 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:24:17.625230 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:24:17.625581 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:24:17.625896 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:24:17.625899 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:24:17.625901 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:24:17.626070 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:24:17.626551 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:24:17.626557 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:24:17.626874 (+ 317us) client_negotiation.cc:769] Sending connection context
0808 18:24:17.627923 (+ 1049us) client_negotiation.cc:240] Negotiation successful
0808 18:24:17.628263 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6379,"thread_start_us":252,"threads_started":1}
I20260808 18:24:17.631291 4157 heartbeater.cc:347] Connected to a master server at 127.23.72.190:40295
I20260808 18:24:17.631572 4157 heartbeater.cc:464] Registering TS with master...
I20260808 18:24:17.632109 4157 heartbeater.cc:511] Master 127.23.72.190:40295 requested a full tablet report, sending...
I20260808 18:24:17.633244 3639 ts_manager.cc:194] Registered new tserver with Master: 6fba1af734cc40849b5e3bbf21a01425 (127.23.72.131:42699)
I20260808 18:24:17.633872 3639 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.131@KRBTEST.COM'} at 127.23.72.131:39445
I20260808 18:24:17.644500 23842 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:24:17.661198 3871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:24:17.651864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60274 (local address 127.23.72.190:40295)
0808 18:24:17.652035 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 18:24:17.652042 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:24:17.652458 (+ 416us) server_negotiation.cc:407] Connection header received
0808 18:24:17.652602 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:24:17.652606 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:24:17.652658 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:24:17.652765 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:24:17.653591 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:24:17.654353 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:24:17.655587 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:24:17.655892 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:24:17.655996 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:24:17.657712 (+ 1716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:24:17.657716 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:24:17.657719 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:24:17.657744 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:24:17.659482 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:24:17.659954 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:24:17.659957 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:24:17.659959 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:24:17.660047 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:24:17.660349 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:24:17.660352 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:24:17.660353 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:24:17.660559 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:24:17.660686 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 18:24:17.660937 (+ 251us) server_negotiation.cc:299] Negotiation successful
0808 18:24:17.661049 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 18:24:17.664742 3639 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60274:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:24:17.667953 3639 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:24:17.685154 4174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:24:17.677535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:42449 (local address 127.0.0.1:50618)
0808 18:24:17.678613 (+ 1078us) negotiation.cc:107] Waiting for socket to connect
0808 18:24:17.678663 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 18:24:17.678788 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:24:17.680237 (+ 1449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:24:17.680240 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:24:17.680257 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:24:17.680476 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:24:17.680486 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:24:17.681961 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:24:17.681964 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:24:17.683350 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:24:17.683360 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:24:17.684542 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:24:17.684546 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:24:17.684711 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:24:17.684769 (+ 58us) client_negotiation.cc:769] Sending connection context
0808 18:24:17.684864 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 18:24:17.684979 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":875,"thread_start_us":398,"threads_started":1}
I20260808 18:24:17.685961 4175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:24:17.679073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.130:37803 (local address 127.0.0.1:47254)
0808 18:24:17.679820 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0808 18:24:17.679833 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:24:17.679943 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:24:17.681213 (+ 1270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:24:17.681216 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:24:17.681232 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:24:17.681386 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:24:17.681393 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:24:17.682879 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:24:17.682882 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:24:17.684038 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:24:17.684048 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:24:17.685499 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:24:17.685502 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:24:17.685655 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:24:17.685678 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 18:24:17.685752 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 18:24:17.685823 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":680,"mutex_wait_us":1,"thread_start_us":146,"threads_started":1}
I20260808 18:24:17.686367 4171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:24:17.677668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50618 (local address 127.23.72.129:42449)
0808 18:24:17.679757 (+ 2089us) server_negotiation.cc:206] Beginning negotiation
0808 18:24:17.679764 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:24:17.679787 (+ 23us) server_negotiation.cc:407] Connection header received
0808 18:24:17.679878 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:24:17.679887 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:24:17.680101 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:24:17.680251 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:24:17.680631 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:24:17.681831 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:24:17.683524 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:24:17.684405 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:24:17.685889 (+ 1484us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:24:17.686042 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 18:24:17.686104 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 18:24:17.686187 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1944,"thread_start_us":191,"threads_started":1}
I20260808 18:24:17.686766 4176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:24:17.679284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47254 (local address 127.23.72.130:37803)
0808 18:24:17.680579 (+ 1295us) server_negotiation.cc:206] Beginning negotiation
0808 18:24:17.680620 (+ 41us) server_negotiation.cc:399] Waiting for connection header
0808 18:24:17.680643 (+ 23us) server_negotiation.cc:407] Connection header received
0808 18:24:17.680726 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:24:17.680730 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:24:17.680907 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:24:17.681080 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:24:17.681517 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:24:17.682747 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:24:17.684252 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:24:17.685163 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:24:17.685275 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:24:17.685439 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 18:24:17.686458 (+ 1019us) server_negotiation.cc:299] Negotiation successful
0808 18:24:17.686561 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1175,"thread_start_us":185,"threads_started":1}
I20260808 18:24:17.690538 3934 tablet_service.cc:1467] Processing CreateTablet for tablet 0fe4dc7ebdbe453ea31d0f87b10d9f54 (DEFAULT_TABLE table=test-table [id=0b04bec30bf345af9d96f9b56542d852]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:24:17.690519 3792 tablet_service.cc:1467] Processing CreateTablet for tablet 0fe4dc7ebdbe453ea31d0f87b10d9f54 (DEFAULT_TABLE table=test-table [id=0b04bec30bf345af9d96f9b56542d852]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:24:17.691884 3934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fe4dc7ebdbe453ea31d0f87b10d9f54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:24:17.691917 3792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fe4dc7ebdbe453ea31d0f87b10d9f54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:24:17.693202 4173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:24:17.677977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33618 (local address 127.23.72.131:42699)
0808 18:24:17.678501 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0808 18:24:17.678540 (+ 39us) server_negotiation.cc:399] Waiting for connection header
0808 18:24:17.679465 (+ 925us) server_negotiation.cc:407] Connection header received
0808 18:24:17.679609 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:24:17.679615 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:24:17.679827 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:24:17.680028 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:24:17.680527 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:24:17.682119 (+ 1592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:24:17.688182 (+ 6063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:24:17.689802 (+ 1620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:24:17.689939 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:24:17.690183 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0808 18:24:17.692644 (+ 2461us) server_negotiation.cc:299] Negotiation successful
0808 18:24:17.692845 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":138,"threads_started":1}
I20260808 18:24:17.693859 4172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:24:17.677602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.131:42699 (local address 127.0.0.1:33618)
0808 18:24:17.679100 (+ 1498us) negotiation.cc:107] Waiting for socket to connect
0808 18:24:17.679113 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:24:17.679484 (+ 371us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:24:17.680206 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:24:17.680212 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:24:17.680235 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:24:17.680374 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:24:17.680380 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:24:17.684353 (+ 3973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:24:17.684356 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:24:17.686467 (+ 2111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:24:17.686486 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:24:17.690463 (+ 3977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:24:17.690467 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:24:17.690797 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:24:17.690894 (+ 97us) client_negotiation.cc:769] Sending connection context
0808 18:24:17.693464 (+ 2570us) client_negotiation.cc:240] Negotiation successful
0808 18:24:17.693606 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1433,"spinlock_wait_cycles":4480,"thread_start_us":633,"threads_started":1}
I20260808 18:24:17.703157 4178 tablet_bootstrap.cc:492] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P b27e7586d9f04315ae3f607bf1a10343: Bootstrap starting.
I20260808 18:24:17.703202 4177 tablet_bootstrap.cc:492] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P a0b0d83253554fe78a3fec33f2267ceb: Bootstrap starting.
I20260808 18:24:17.706312 4177 tablet_bootstrap.cc:654] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P a0b0d83253554fe78a3fec33f2267ceb: Neither blocks nor log segments found. Creating new log.
I20260808 18:24:17.698782 4083 tablet_service.cc:1467] Processing CreateTablet for tablet 0fe4dc7ebdbe453ea31d0f87b10d9f54 (DEFAULT_TABLE table=test-table [id=0b04bec30bf345af9d96f9b56542d852]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:24:17.708427 4083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fe4dc7ebdbe453ea31d0f87b10d9f54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:24:17.716377 4177 log.cc:824] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P a0b0d83253554fe78a3fec33f2267ceb: Log is configured to *not* fsync() on all Append() calls
I20260808 18:24:17.719626 4178 tablet_bootstrap.cc:654] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P b27e7586d9f04315ae3f607bf1a10343: Neither blocks nor log segments found. Creating new log.
I20260808 18:24:17.721241 4178 log.cc:824] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P b27e7586d9f04315ae3f607bf1a10343: Log is configured to *not* fsync() on all Append() calls
I20260808 18:24:17.724623 4178 tablet_bootstrap.cc:492] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P b27e7586d9f04315ae3f607bf1a10343: No bootstrap required, opened a new log
I20260808 18:24:17.724931 4178 ts_tablet_manager.cc:1397] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P b27e7586d9f04315ae3f607bf1a10343: Time spent bootstrapping tablet: real 0.022s user 0.007s sys 0.000s
I20260808 18:24:17.726646 4177 tablet_bootstrap.cc:492] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P a0b0d83253554fe78a3fec33f2267ceb: No bootstrap required, opened a new log
I20260808 18:24:17.726912 4177 ts_tablet_manager.cc:1397] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P a0b0d83253554fe78a3fec33f2267ceb: Time spent bootstrapping tablet: real 0.024s user 0.006s sys 0.000s
I20260808 18:24:17.730988 4179 tablet_bootstrap.cc:492] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P 6fba1af734cc40849b5e3bbf21a01425: Bootstrap starting.
I20260808 18:24:17.732267 4178 raft_consensus.cc:348] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P b27e7586d9f04315ae3f607bf1a10343 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0b0d83253554fe78a3fec33f2267ceb" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 42449 } } peers { permanent_uuid: "6fba1af734cc40849b5e3bbf21a01425" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 42699 } } peers { permanent_uuid: "b27e7586d9f04315ae3f607bf1a10343" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 37803 } }
I20260808 18:24:17.732267 4177 raft_consensus.cc:348] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P a0b0d83253554fe78a3fec33f2267ceb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0b0d83253554fe78a3fec33f2267ceb" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 42449 } } peers { permanent_uuid: "6fba1af734cc40849b5e3bbf21a01425" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 42699 } } peers { permanent_uuid: "b27e7586d9f04315ae3f607bf1a10343" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 37803 } }
I20260808 18:24:17.732549 4177 raft_consensus.cc:374] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P a0b0d83253554fe78a3fec33f2267ceb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:24:17.732623 4177 raft_consensus.cc:729] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P a0b0d83253554fe78a3fec33f2267ceb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0b0d83253554fe78a3fec33f2267ceb, State: Initialized, Role: FOLLOWER
I20260808 18:24:17.732684 4178 raft_consensus.cc:374] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P b27e7586d9f04315ae3f607bf1a10343 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:24:17.732782 4178 raft_consensus.cc:729] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P b27e7586d9f04315ae3f607bf1a10343 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b27e7586d9f04315ae3f607bf1a10343, State: Initialized, Role: FOLLOWER
I20260808 18:24:17.733217 4177 consensus_queue.cc:260] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P a0b0d83253554fe78a3fec33f2267ceb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0b0d83253554fe78a3fec33f2267ceb" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 42449 } } peers { permanent_uuid: "6fba1af734cc40849b5e3bbf21a01425" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 42699 } } peers { permanent_uuid: "b27e7586d9f04315ae3f607bf1a10343" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 37803 } }
I20260808 18:24:17.733283 4178 consensus_queue.cc:260] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P b27e7586d9f04315ae3f607bf1a10343 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0b0d83253554fe78a3fec33f2267ceb" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 42449 } } peers { permanent_uuid: "6fba1af734cc40849b5e3bbf21a01425" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 42699 } } peers { permanent_uuid: "b27e7586d9f04315ae3f607bf1a10343" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 37803 } }
I20260808 18:24:17.734424 4179 tablet_bootstrap.cc:654] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P 6fba1af734cc40849b5e3bbf21a01425: Neither blocks nor log segments found. Creating new log.
I20260808 18:24:17.734549 4177 ts_tablet_manager.cc:1428] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P a0b0d83253554fe78a3fec33f2267ceb: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260808 18:24:17.734549 4178 ts_tablet_manager.cc:1428] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P b27e7586d9f04315ae3f607bf1a10343: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260808 18:24:17.734946 3868 heartbeater.cc:503] Master 127.23.72.190:40295 was elected leader, sending a full tablet report...
I20260808 18:24:17.735239 4017 heartbeater.cc:503] Master 127.23.72.190:40295 was elected leader, sending a full tablet report...
I20260808 18:24:17.736989 4179 log.cc:824] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P 6fba1af734cc40849b5e3bbf21a01425: Log is configured to *not* fsync() on all Append() calls
I20260808 18:24:17.740439 4179 tablet_bootstrap.cc:492] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P 6fba1af734cc40849b5e3bbf21a01425: No bootstrap required, opened a new log
I20260808 18:24:17.740666 4179 ts_tablet_manager.cc:1397] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P 6fba1af734cc40849b5e3bbf21a01425: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260808 18:24:17.746819 4179 raft_consensus.cc:348] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P 6fba1af734cc40849b5e3bbf21a01425 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0b0d83253554fe78a3fec33f2267ceb" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 42449 } } peers { permanent_uuid: "6fba1af734cc40849b5e3bbf21a01425" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 42699 } } peers { permanent_uuid: "b27e7586d9f04315ae3f607bf1a10343" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 37803 } }
I20260808 18:24:17.747218 4179 raft_consensus.cc:374] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P 6fba1af734cc40849b5e3bbf21a01425 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:24:17.747354 4179 raft_consensus.cc:729] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P 6fba1af734cc40849b5e3bbf21a01425 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fba1af734cc40849b5e3bbf21a01425, State: Initialized, Role: FOLLOWER
I20260808 18:24:17.747938 4179 consensus_queue.cc:260] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P 6fba1af734cc40849b5e3bbf21a01425 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0b0d83253554fe78a3fec33f2267ceb" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 42449 } } peers { permanent_uuid: "6fba1af734cc40849b5e3bbf21a01425" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 42699 } } peers { permanent_uuid: "b27e7586d9f04315ae3f607bf1a10343" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 37803 } }
I20260808 18:24:17.748860 4157 heartbeater.cc:503] Master 127.23.72.190:40295 was elected leader, sending a full tablet report...
I20260808 18:24:17.749364 4179 ts_tablet_manager.cc:1428] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P 6fba1af734cc40849b5e3bbf21a01425: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
W20260808 18:24:17.857118 4160 tablet.cc:2369] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P 6fba1af734cc40849b5e3bbf21a01425: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 18:24:17 dist-test-slave-5k54 krb5kdc[3124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213456, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for HTTP/127.23.72.148@KRBTEST.COM
I20260808 18:24:17.915827 4185 raft_consensus.cc:482] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P 6fba1af734cc40849b5e3bbf21a01425 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:24:17.916152 4185 raft_consensus.cc:504] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P 6fba1af734cc40849b5e3bbf21a01425 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0b0d83253554fe78a3fec33f2267ceb" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 42449 } } peers { permanent_uuid: "6fba1af734cc40849b5e3bbf21a01425" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 42699 } } peers { permanent_uuid: "b27e7586d9f04315ae3f607bf1a10343" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 37803 } }
I20260808 18:24:17.922842 4185 leader_election.cc:288] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P 6fba1af734cc40849b5e3bbf21a01425 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0b0d83253554fe78a3fec33f2267ceb (127.23.72.129:42449), b27e7586d9f04315ae3f607bf1a10343 (127.23.72.130:37803)
I20260808 18:24:17.923969 4161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:24:17.918225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:42449 (local address 127.23.72.131:54447)
0808 18:24:17.918776 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0808 18:24:17.918793 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 18:24:17.918892 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:24:17.919621 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:24:17.919624 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:24:17.919645 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:24:17.919839 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:24:17.919847 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:24:17.920907 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:24:17.920910 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:24:17.922253 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:24:17.922275 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:24:17.923401 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:24:17.923405 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:24:17.923597 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:24:17.923624 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 18:24:17.923705 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 18:24:17.923815 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455}
I20260808 18:24:17.926366 4171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:24:17.919086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:54447 (local address 127.23.72.129:42449)
0808 18:24:17.919209 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 18:24:17.919213 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:24:17.919228 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:24:17.919295 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:24:17.919298 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:24:17.919348 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:24:17.919453 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:24:17.919982 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:24:17.920773 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:24:17.922433 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:24:17.923135 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:24:17.923267 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:24:17.923327 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 18:24:17.926106 (+ 2779us) server_negotiation.cc:299] Negotiation successful
0808 18:24:17.926217 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 18:24:17.927218 3819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe4dc7ebdbe453ea31d0f87b10d9f54" candidate_uuid: "6fba1af734cc40849b5e3bbf21a01425" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0b0d83253554fe78a3fec33f2267ceb" is_pre_election: true
I20260808 18:24:17.927639 3819 raft_consensus.cc:2454] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P a0b0d83253554fe78a3fec33f2267ceb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fba1af734cc40849b5e3bbf21a01425 in term 0.
I20260808 18:24:17.928745 4039 leader_election.cc:302] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P 6fba1af734cc40849b5e3bbf21a01425 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fba1af734cc40849b5e3bbf21a01425, a0b0d83253554fe78a3fec33f2267ceb; no voters:
I20260808 18:24:17.929251 4185 raft_consensus.cc:2781] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P 6fba1af734cc40849b5e3bbf21a01425 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:24:17.929350 4185 raft_consensus.cc:482] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P 6fba1af734cc40849b5e3bbf21a01425 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:24:17.929392 4185 raft_consensus.cc:3037] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P 6fba1af734cc40849b5e3bbf21a01425 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:24:17.931403 4161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:24:17.923177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.130:37803 (local address 127.23.72.131:37957)
0808 18:24:17.924135 (+ 958us) negotiation.cc:107] Waiting for socket to connect
0808 18:24:17.924150 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:24:17.924236 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:24:17.926571 (+ 2335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:24:17.926588 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:24:17.926611 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:24:17.926699 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:24:17.926707 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:24:17.927978 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:24:17.927981 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:24:17.929425 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:24:17.929438 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:24:17.930850 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:24:17.930857 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:24:17.930957 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:24:17.930982 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 18:24:17.931052 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 18:24:17.931122 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":900,"thread_start_us":165,"threads_started":1}
I20260808 18:24:17.931779 4185 raft_consensus.cc:504] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P 6fba1af734cc40849b5e3bbf21a01425 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0b0d83253554fe78a3fec33f2267ceb" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 42449 } } peers { permanent_uuid: "6fba1af734cc40849b5e3bbf21a01425" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 42699 } } peers { permanent_uuid: "b27e7586d9f04315ae3f607bf1a10343" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 37803 } }
I20260808 18:24:17.931850 4176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:24:17.923630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:37957 (local address 127.23.72.130:37803)
0808 18:24:17.923747 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 18:24:17.923752 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:24:17.925303 (+ 1551us) server_negotiation.cc:407] Connection header received
0808 18:24:17.925390 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:24:17.925394 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:24:17.925452 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:24:17.925558 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:24:17.926872 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:24:17.927833 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:24:17.929603 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:24:17.930483 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:24:17.930644 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:24:17.930717 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 18:24:17.931596 (+ 879us) server_negotiation.cc:299] Negotiation successful
0808 18:24:17.931683 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 18:24:17.932520 3963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe4dc7ebdbe453ea31d0f87b10d9f54" candidate_uuid: "6fba1af734cc40849b5e3bbf21a01425" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b27e7586d9f04315ae3f607bf1a10343" is_pre_election: true
I20260808 18:24:17.932849 3963 raft_consensus.cc:2454] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P b27e7586d9f04315ae3f607bf1a10343 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fba1af734cc40849b5e3bbf21a01425 in term 0.
I20260808 18:24:17.933434 3960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe4dc7ebdbe453ea31d0f87b10d9f54" candidate_uuid: "6fba1af734cc40849b5e3bbf21a01425" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b27e7586d9f04315ae3f607bf1a10343"
I20260808 18:24:17.933548 3819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe4dc7ebdbe453ea31d0f87b10d9f54" candidate_uuid: "6fba1af734cc40849b5e3bbf21a01425" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0b0d83253554fe78a3fec33f2267ceb"
I20260808 18:24:17.933565 3960 raft_consensus.cc:3037] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P b27e7586d9f04315ae3f607bf1a10343 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:24:17.933670 3819 raft_consensus.cc:3037] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P a0b0d83253554fe78a3fec33f2267ceb [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:24:17.935299 3819 raft_consensus.cc:2454] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P a0b0d83253554fe78a3fec33f2267ceb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fba1af734cc40849b5e3bbf21a01425 in term 1.
I20260808 18:24:17.935524 3960 raft_consensus.cc:2454] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P b27e7586d9f04315ae3f607bf1a10343 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fba1af734cc40849b5e3bbf21a01425 in term 1.
I20260808 18:24:17.935658 4039 leader_election.cc:302] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P 6fba1af734cc40849b5e3bbf21a01425 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fba1af734cc40849b5e3bbf21a01425, a0b0d83253554fe78a3fec33f2267ceb; no voters:
I20260808 18:24:17.936087 4185 leader_election.cc:288] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P 6fba1af734cc40849b5e3bbf21a01425 [CANDIDATE]: Term 1 election: Requested vote from peers a0b0d83253554fe78a3fec33f2267ceb (127.23.72.129:42449), b27e7586d9f04315ae3f607bf1a10343 (127.23.72.130:37803)
I20260808 18:24:17.936232 4185 raft_consensus.cc:2781] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P 6fba1af734cc40849b5e3bbf21a01425 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:24:17.936321 4185 raft_consensus.cc:686] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P 6fba1af734cc40849b5e3bbf21a01425 [term 1 LEADER]: Becoming Leader. State: Replica: 6fba1af734cc40849b5e3bbf21a01425, State: Running, Role: LEADER
I20260808 18:24:17.936651 4185 consensus_queue.cc:237] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P 6fba1af734cc40849b5e3bbf21a01425 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0b0d83253554fe78a3fec33f2267ceb" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 42449 } } peers { permanent_uuid: "6fba1af734cc40849b5e3bbf21a01425" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 42699 } } peers { permanent_uuid: "b27e7586d9f04315ae3f607bf1a10343" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 37803 } }
W20260808 18:24:17.938360 4018 tablet.cc:2369] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P b27e7586d9f04315ae3f607bf1a10343: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:24:17.940138 3639 catalog_manager.cc:5495] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P 6fba1af734cc40849b5e3bbf21a01425 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6fba1af734cc40849b5e3bbf21a01425 (127.23.72.131). New cstate: current_term: 1 leader_uuid: "6fba1af734cc40849b5e3bbf21a01425" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0b0d83253554fe78a3fec33f2267ceb" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 42449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fba1af734cc40849b5e3bbf21a01425" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 42699 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b27e7586d9f04315ae3f607bf1a10343" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 37803 } health_report { overall_health: UNKNOWN } } }
W20260808 18:24:17.948845 3869 tablet.cc:2369] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P a0b0d83253554fe78a3fec33f2267ceb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:24:17.996110 4173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:24:17.990266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33626 (local address 127.23.72.131:42699)
0808 18:24:17.990465 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 18:24:17.990469 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:24:17.990485 (+ 16us) server_negotiation.cc:407] Connection header received
0808 18:24:17.990611 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:24:17.990623 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:24:17.990727 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:24:17.990883 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:24:17.991638 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:24:17.992419 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:24:17.993649 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:24:17.993987 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:24:17.994091 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:24:17.994620 (+ 529us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:24:17.995205 (+ 585us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:24:17.995325 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 18:24:17.995798 (+ 473us) server_negotiation.cc:299] Negotiation successful
0808 18:24:17.995860 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 18:24:18.003973 3960 raft_consensus.cc:1261] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P b27e7586d9f04315ae3f607bf1a10343 [term 1 FOLLOWER]: Refusing update from remote peer 6fba1af734cc40849b5e3bbf21a01425: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:24:18.005075 4190 consensus_queue.cc:1035] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P 6fba1af734cc40849b5e3bbf21a01425 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b27e7586d9f04315ae3f607bf1a10343" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 37803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:24:18.014204 3819 raft_consensus.cc:1261] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P a0b0d83253554fe78a3fec33f2267ceb [term 1 FOLLOWER]: Refusing update from remote peer 6fba1af734cc40849b5e3bbf21a01425: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:24:18.014984 4190 consensus_queue.cc:1035] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P 6fba1af734cc40849b5e3bbf21a01425 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0b0d83253554fe78a3fec33f2267ceb" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 42449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:24:18.031567 4194 mvcc.cc:205] Tried to move back new op lower bound from 7316330323974844416 to 7316330323711229952. Current Snapshot: MvccSnapshot[applied={T|T < 7316330323974844416}]
I20260808 18:24:18.039772 3638 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60274:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:24:18.040002 3638 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60274:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:24:18.042709 3638 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 59b31f9e14ec40508b714b59729ce179: Sending DeleteTablet for 3 replicas of tablet 0fe4dc7ebdbe453ea31d0f87b10d9f54
I20260808 18:24:18.043733 3792 tablet_service.cc:1514] Processing DeleteTablet for tablet 0fe4dc7ebdbe453ea31d0f87b10d9f54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:24:18 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:50618
I20260808 18:24:18.043781 3934 tablet_service.cc:1514] Processing DeleteTablet for tablet 0fe4dc7ebdbe453ea31d0f87b10d9f54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:24:18 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:47254
I20260808 18:24:18.044414 4083 tablet_service.cc:1514] Processing DeleteTablet for tablet 0fe4dc7ebdbe453ea31d0f87b10d9f54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:24:18 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:33618
I20260808 18:24:18.044915 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 3725
I20260808 18:24:18.051170 4202 tablet_replica.cc:330] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P b27e7586d9f04315ae3f607bf1a10343: stopping tablet replica
I20260808 18:24:18.051746 4202 raft_consensus.cc:2229] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P b27e7586d9f04315ae3f607bf1a10343 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:24:18.052139 4202 raft_consensus.cc:2258] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P b27e7586d9f04315ae3f607bf1a10343 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 18:24:18.058930 4039 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.72.129:42449 (error 108)
I20260808 18:24:18.059122 4203 tablet_replica.cc:330] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P 6fba1af734cc40849b5e3bbf21a01425: stopping tablet replica
I20260808 18:24:18.059629 4203 raft_consensus.cc:2229] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P 6fba1af734cc40849b5e3bbf21a01425 [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:24:18.060093 4203 raft_consensus.cc:2258] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P 6fba1af734cc40849b5e3bbf21a01425 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:24:18.061151 4202 ts_tablet_manager.cc:1905] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P b27e7586d9f04315ae3f607bf1a10343: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:24:18.064146 4202 ts_tablet_manager.cc:1918] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P b27e7586d9f04315ae3f607bf1a10343: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:24:18.064251 4202 log.cc:1196] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P b27e7586d9f04315ae3f607bf1a10343: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-1/wal/wals/0fe4dc7ebdbe453ea31d0f87b10d9f54
I20260808 18:24:18.064599 4202 ts_tablet_manager.cc:1939] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P b27e7586d9f04315ae3f607bf1a10343: Deleting consensus metadata
I20260808 18:24:18.066102 3622 catalog_manager.cc:4841] TS b27e7586d9f04315ae3f607bf1a10343 (127.23.72.130:37803): tablet 0fe4dc7ebdbe453ea31d0f87b10d9f54 (table test-table [id=0b04bec30bf345af9d96f9b56542d852]) successfully deleted
W20260808 18:24:18.066620 3623 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.72.129:42449 (error 108)
I20260808 18:24:18.067643 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 3877
I20260808 18:24:18.069341 4172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:24:18.068986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:42449 (local address 127.0.0.1:50620)
0808 18:24:18.069183 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 18:24:18.069252 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.129:42449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
W20260808 18:24:18.069527 3623 catalog_manager.cc:4584] TS a0b0d83253554fe78a3fec33f2267ceb (127.23.72.129:42449): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0fe4dc7ebdbe453ea31d0f87b10d9f54: Network error: Client connection negotiation failed: client connection to 127.23.72.129:42449: connect: Connection refused (error 111)
I20260808 18:24:18.071075 4203 ts_tablet_manager.cc:1905] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P 6fba1af734cc40849b5e3bbf21a01425: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:24:18.074666 4203 ts_tablet_manager.cc:1918] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P 6fba1af734cc40849b5e3bbf21a01425: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:24:18.074792 4203 log.cc:1196] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P 6fba1af734cc40849b5e3bbf21a01425: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213345431840-23842-0/minicluster-data/ts-2/wal/wals/0fe4dc7ebdbe453ea31d0f87b10d9f54
I20260808 18:24:18.075135 4203 ts_tablet_manager.cc:1939] T 0fe4dc7ebdbe453ea31d0f87b10d9f54 P 6fba1af734cc40849b5e3bbf21a01425: Deleting consensus metadata
I20260808 18:24:18.076627 3625 catalog_manager.cc:4841] TS 6fba1af734cc40849b5e3bbf21a01425 (127.23.72.131:42699): tablet 0fe4dc7ebdbe453ea31d0f87b10d9f54 (table test-table [id=0b04bec30bf345af9d96f9b56542d852]) successfully deleted
I20260808 18:24:18.077896 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 4022
I20260808 18:24:18.096297 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 3604
I20260808 18:24:18.105202 23842 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 18:24:18.460661 23842 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 18:24:18.460793 23842 mini_ranger.cc:66] Stopping Ranger...
I20260808 18:24:18.805666 23842 mini_ranger.cc:68] Stopped Ranger
2026-08-08 18:24:18.805 UTC [3154] LOG: received smart shutdown request
2026-08-08 18:24:18.809 UTC [3154] LOG: background worker "logical replication launcher" (PID 3163) exited with exit code 1
2026-08-08 18:24:18.810 UTC [3158] LOG: shutting down
2026-08-08 18:24:19.006 UTC [3154] LOG: database system is shut down
2026-08-08T18:24:19Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (49394 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:24:19 dist-test-slave-5k54 krb5kdc[4215](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:24:19 dist-test-slave-5k54 krb5kdc[4215](info): set up 2 sockets
Aug 08 18:24:19 dist-test-slave-5k54 krb5kdc[4215](info): commencing operation
krb5kdc: starting...
W20260808 18:24:20.224292 23842 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Aug 08 18:24:20 dist-test-slave-5k54 krb5kdc[4215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213460, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:24:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:24:20Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.23.72.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.23.72.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.23.72.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/rangeradmin_127.23.72.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.23.72.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/rangeradmin_127.23.72.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.23.72.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.23.72.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.23.72.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/rangerlookup_127.23.72.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.23.72.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/rangerlookup_127.23.72.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.23.72.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.23.72.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.23.72.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/HTTP_127.23.72.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.23.72.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/HTTP_127.23.72.148@KRBTEST.COM.keytab.
I20260808 18:24:20.403486 23842 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskVyYox2/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/postgres -l logfile start
2026-08-08 18:24:22.923 UTC [4248] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 18:24:22.923 UTC [4248] LOG: listening on IPv4 address "127.23.72.148", port 49737
2026-08-08 18:24:22.927 UTC [4248] LOG: listening on Unix socket "/tmp/.s.PGSQL.49737"
2026-08-08 18:24:22.947 UTC [4251] LOG: database system was shut down at 2026-08-08 18:24:21 UTC
2026-08-08 18:24:22.951 UTC [4248] LOG: database system is ready to accept connections
I20260808 18:24:23.940344 23842 mini_postgres.cc:96] Postgres bound to 49737
2026-08-08 18:24:23.947 UTC [4260] FATAL: database "slave" does not exist
127.23.72.148:49737 - accepting connections
I20260808 18:24:23.948343 23842 mini_ranger.cc:161] Starting Ranger...
I20260808 18:24:23.964283 23842 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 18:24:24.440789 23842 mini_ranger.cc:87] Created ranger Postgres database
I20260808 18:24:24.440899 23842 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ranger-admin
2026-08-08 18:24:24,714 [I] DB FLAVOR :POSTGRES
2026-08-08 18:24:24,715 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 18:24:24,715 [I] Checking connection..
2026-08-08 18:24:24,715 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.72.148:49737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 18:24:24,967 [I] Checking connection passed.
2026-08-08 18:24:24,967 [I] --------- Verifying version history table ---------
2026-08-08 18:24:24,968 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.72.148:49737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 18:24:25,235 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 18:24:25,235 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.72.148:49737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 18:24:25,512 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 18:24:25,512 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 18:24:25,512 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.72.148:49737/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 18:24:25.761 UTC [4332] WARNING: there is no transaction in progress
2026-08-08 18:24:25,768 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 18:24:25,769 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.72.148:49737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 18:24:26,020 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 18:24:26,020 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 18:24:26,020 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.72.148:49737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 18:24:26,284 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.72.148:49737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 18:24:26,538 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.72.148:49737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-5k54.c.gcp-upstream.internal','N') ;"
2026-08-08 18:24:26,784 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.72.148:49737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 18:24:27,039 [I] Table x_portal_user does not exist in database ranger
2026-08-08 18:24:27,040 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.72.148:49737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 18:24:27,303 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 18:24:27,304 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.72.148:49737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 18:24:27,557 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.72.148:49737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 18:24:27,818 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 18:24:27,818 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.72.148:49737/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 18:24:28.844 UTC [4449] WARNING: there is no transaction in progress
2026-08-08 18:24:28.863 UTC [4449] WARNING: there is no transaction in progress
2026-08-08 18:24:28.882 UTC [4449] WARNING: there is no transaction in progress
2026-08-08 18:24:28.897 UTC [4449] WARNING: there is no transaction in progress
2026-08-08 18:24:28.913 UTC [4449] WARNING: there is no transaction in progress
2026-08-08 18:24:28.929 UTC [4449] WARNING: there is no transaction in progress
2026-08-08 18:24:28.985 UTC [4449] WARNING: there is no transaction in progress
2026-08-08 18:24:28.992 UTC [4449] WARNING: there is no transaction in progress
2026-08-08 18:24:29.007 UTC [4449] WARNING: there is no transaction in progress
2026-08-08 18:24:29.018 UTC [4449] WARNING: there is no transaction in progress
2026-08-08 18:24:29.028 UTC [4449] WARNING: there is no transaction in progress
2026-08-08 18:24:29.042 UTC [4449] WARNING: there is no transaction in progress
2026-08-08 18:24:29.053 UTC [4449] WARNING: there is no transaction in progress
2026-08-08 18:24:29.059 UTC [4449] WARNING: there is no transaction in progress
2026-08-08 18:24:29.066 UTC [4449] WARNING: there is no transaction in progress
2026-08-08 18:24:29.073 UTC [4449] WARNING: there is no transaction in progress
2026-08-08 18:24:29,734 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 18:24:29,735 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.72.148:49737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 18:24:29,983 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 18:24:29,984 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.72.148:49737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 18:24:30,241 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 18:24:30,241 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.72.148:49737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 18:24:30,504 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 18:24:30,505 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.72.148:49737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 18:24:30,771 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.72.148:49737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 18:24:31,032 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.72.148:49737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-5k54.c.gcp-upstream.internal';"
2026-08-08 18:24:31,283 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 18:24:31,283 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.72.148:49737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 18:24:31,544 [I] DB_PATCHES have already been applied
I20260808 18:24:31.550208 23842 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ranger-admin:/tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskVyYox2/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 18:24:31.550274 23842 mini_ranger.cc:193] Using host: 127.23.72.148
I20260808 18:24:31.554628 23842 mini_ranger.cc:227] Ranger admin URL: http://127.23.72.148:33915
Aug 08, 2026 6:24:31 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 6:24:31 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 6:24:31 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 6:24:31 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 6:24:31 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 6:24:32 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 6:24:32 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 18:24:32.699159 23842 mini_ranger.cc:160] Time spent starting Ranger: real 8.751s user 0.000s sys 0.012s
Aug 08, 2026 6:24:53 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 6:24:53 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 08, 2026 6:24:53 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 6:24:53 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 6:24:53 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 6:24:53 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 6:24:53 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 6:24:53 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 6:24:53 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 6:24:53 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 6:24:53 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 6:24:53 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 6:24:53 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 6:24:53 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 6:24:53 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 6:24:53 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 6:24:53 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 6:24:54 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 18:24:54.732053 23842 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.23.72.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.23.72.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.23.72.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/rangerkms_127.23.72.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.23.72.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/rangerkms_127.23.72.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.23.72.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.23.72.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.23.72.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/HTTP_127.23.72.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.23.72.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/HTTP_127.23.72.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 18:24:54 dist-test-slave-5k54 krb5kdc[4215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213494, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 18:24:54.827992 23842 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 18:24:54.838693 23842 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 18:24:55.780293 23842 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 18:24:55.780390 23842 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ranger-kms
I20260808 18:24:55.780413 23842 mini_ranger_kms.cc:227] Using postgres at 127.23.72.148:49737
2026-08-08 18:24:55,955 [I] DB FLAVOR :POSTGRES
2026-08-08 18:24:55,956 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 18:24:55,956 [I] Checking connection
2026-08-08 18:24:55,956 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.72.148:49737/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 18:24:56,205 [I] connection success
2026-08-08 18:24:56,205 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 18:24:56,205 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 18:24:56,205 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.72.148:49737/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 18:24:56,446 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 18:24:56,447 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 18:24:56,447 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 18:24:56,447 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.72.148:49737/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 18:24:56,714 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 18:24:57.190877 23842 mini_ranger_kms.cc:310] Created kms service
I20260808 18:24:57.362363 23842 mini_ranger_kms.cc:326] Created kudu user
I20260808 18:24:57.407393 23842 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 18:24:57 dist-test-slave-5k54 krb5kdc[4215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213494, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.23.72.148@KRBTEST.COM
I20260808 18:24:57.808079 23842 mini_ranger_kms.cc:384] Added ranger policy
I20260808 18:24:57.808233 23842 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ranger-kms:/tmp/dist-test-taskVyYox2/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskVyYox2/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 18:24:57.808251 23842 mini_ranger_kms.cc:240] Using host: 127.23.72.148
I20260808 18:24:57.813659 23842 mini_ranger_kms.cc:276] Ranger KMS PID: 4636
I20260808 18:24:57.813766 23842 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.23.72.148:54221
Aug 08, 2026 6:24:58 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 6:24:58 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 6:24:58 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 6:24:58 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 6:24:59 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 6:24:59 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 18:25:00 dist-test-slave-5k54 krb5kdc[4215](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786213500, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.23.72.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:25:00.224458 23842 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.396s user 0.003s sys 0.158s
I20260808 18:25:00.224608 23842 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 18:25:00.224694 23842 mini_ranger_kms.cc:401] 127.23.72.148:54221/kms/v1/keys
Aug 08 18:25:00 dist-test-slave-5k54 krb5kdc[4215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213500, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.23.72.148@KRBTEST.COM for HTTP/127.23.72.148@KRBTEST.COM
Aug 08 18:25:00 dist-test-slave-5k54 krb5kdc[4215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213500, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.23.72.148@KRBTEST.COM for HTTP/127.23.72.148@KRBTEST.COM
Aug 08 18:25:03 dist-test-slave-5k54 krb5kdc[4215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.23.72.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
I20260808 18:25:03.079358 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:41867
--webserver_interface=127.23.72.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:40107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:41867
--ranger_config_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.72.148:54221/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:25:03.217571 4692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:25:03.217834 4692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:25:03.217877 4692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:25:03.221987 4692 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:25:03.222054 4692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:25:03.222074 4692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:25:03.222090 4692 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:25:03.222105 4692 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:25:03.227928 4692 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:40107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/master-0/wal
--ranger_kms_url=127.23.72.148:54221/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:41867
--ranger_config_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:41867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.72.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:25:03.228991 4692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:25:03.230043 4692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:25:03.240099 4698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:03.240149 4700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:25:03.240306 4692 server_base.cc:1034] running on GCE node
W20260808 18:25:03.240373 4697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:25:03.241080 4692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:25:03.242183 4692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:25:03.243396 4692 hybrid_clock.cc:648] HybridClock initialized: now 1786213503243381 us; error 45 us; skew 500 ppm
Aug 08 18:25:03 dist-test-slave-5k54 krb5kdc[4215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213503, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:25:03.251662 4692 init.cc:377] Logged in from keytab as kudu/127.23.72.190@KRBTEST.COM (short username kudu)
I20260808 18:25:03.252679 4692 webserver.cc:460] Webserver started at http://127.23.72.190:42279/ using document root <none> and password file <none>
I20260808 18:25:03.253260 4692 fs_manager.cc:359] Metadata directory not provided
I20260808 18:25:03.253356 4692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:25:03.253607 4692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 18:25:03 dist-test-slave-5k54 krb5kdc[4215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213503, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for HTTP/127.23.72.148@KRBTEST.COM
I20260808 18:25:03.303097 4692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "874ed22651ea4961bc0120df29e12100"
format_stamp: "Formatted at 2026-08-08 18:25:03 on dist-test-slave-5k54"
server_key: "93b88b5222175c73dfca4828613c5053"
server_key_iv: "95fab39dbcdbd1492941169d55f5fcad"
server_key_version: "kuduclusterkey@0"
I20260808 18:25:03.303769 4692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "874ed22651ea4961bc0120df29e12100"
format_stamp: "Formatted at 2026-08-08 18:25:03 on dist-test-slave-5k54"
server_key: "93b88b5222175c73dfca4828613c5053"
server_key_iv: "95fab39dbcdbd1492941169d55f5fcad"
server_key_version: "kuduclusterkey@0"
I20260808 18:25:03.307888 4692 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 18:25:03.322526 4707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:03.325322 4692 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 18:25:03.325475 4692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "874ed22651ea4961bc0120df29e12100"
format_stamp: "Formatted at 2026-08-08 18:25:03 on dist-test-slave-5k54"
server_key: "93b88b5222175c73dfca4828613c5053"
server_key_iv: "95fab39dbcdbd1492941169d55f5fcad"
server_key_version: "kuduclusterkey@0"
I20260808 18:25:03.325635 4692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:25:03.338174 4692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:25:03.341197 4692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:25:03.341413 4692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:25:03.349871 4692 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.190:41867
I20260808 18:25:03.349900 4764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.190:41867 every 8 connection(s)
I20260808 18:25:03.350850 4692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:25:03.353641 4765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:25:03.358704 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 4692
I20260808 18:25:03.358863 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:25:03.359202 23842 external_mini_cluster.cc:1428] Setting key b992a178083d7659f5e062024b167a79
I20260808 18:25:03.359289 4765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 874ed22651ea4961bc0120df29e12100: Bootstrap starting.
I20260808 18:25:03.362201 4765 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 874ed22651ea4961bc0120df29e12100: Neither blocks nor log segments found. Creating new log.
I20260808 18:25:03.363123 4765 log.cc:824] T 00000000000000000000000000000000 P 874ed22651ea4961bc0120df29e12100: Log is configured to *not* fsync() on all Append() calls
I20260808 18:25:03.365351 4765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 874ed22651ea4961bc0120df29e12100: No bootstrap required, opened a new log
Aug 08 18:25:03 dist-test-slave-5k54 krb5kdc[4215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:25:03.369879 4765 raft_consensus.cc:348] T 00000000000000000000000000000000 P 874ed22651ea4961bc0120df29e12100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "874ed22651ea4961bc0120df29e12100" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 41867 } }
I20260808 18:25:03.370115 4765 raft_consensus.cc:374] T 00000000000000000000000000000000 P 874ed22651ea4961bc0120df29e12100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:25:03.370188 4765 raft_consensus.cc:729] T 00000000000000000000000000000000 P 874ed22651ea4961bc0120df29e12100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 874ed22651ea4961bc0120df29e12100, State: Initialized, Role: FOLLOWER
I20260808 18:25:03.370723 4765 consensus_queue.cc:260] T 00000000000000000000000000000000 P 874ed22651ea4961bc0120df29e12100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "874ed22651ea4961bc0120df29e12100" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 41867 } }
I20260808 18:25:03.370900 4765 raft_consensus.cc:388] T 00000000000000000000000000000000 P 874ed22651ea4961bc0120df29e12100 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:25:03.371007 4765 raft_consensus.cc:482] T 00000000000000000000000000000000 P 874ed22651ea4961bc0120df29e12100 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:25:03.371158 4765 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 874ed22651ea4961bc0120df29e12100 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:25:03.372452 4765 raft_consensus.cc:504] T 00000000000000000000000000000000 P 874ed22651ea4961bc0120df29e12100 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "874ed22651ea4961bc0120df29e12100" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 41867 } }
I20260808 18:25:03.372825 4765 leader_election.cc:302] T 00000000000000000000000000000000 P 874ed22651ea4961bc0120df29e12100 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 874ed22651ea4961bc0120df29e12100; no voters:
I20260808 18:25:03.373194 4765 leader_election.cc:288] T 00000000000000000000000000000000 P 874ed22651ea4961bc0120df29e12100 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:25:03.373289 4770 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 874ed22651ea4961bc0120df29e12100 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:25:03.373576 4770 raft_consensus.cc:686] T 00000000000000000000000000000000 P 874ed22651ea4961bc0120df29e12100 [term 1 LEADER]: Becoming Leader. State: Replica: 874ed22651ea4961bc0120df29e12100, State: Running, Role: LEADER
I20260808 18:25:03.373951 4770 consensus_queue.cc:237] T 00000000000000000000000000000000 P 874ed22651ea4961bc0120df29e12100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "874ed22651ea4961bc0120df29e12100" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 41867 } }
I20260808 18:25:03.374656 4765 sys_catalog.cc:564] T 00000000000000000000000000000000 P 874ed22651ea4961bc0120df29e12100 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:25:03.375234 4770 sys_catalog.cc:455] T 00000000000000000000000000000000 P 874ed22651ea4961bc0120df29e12100 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "874ed22651ea4961bc0120df29e12100" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "874ed22651ea4961bc0120df29e12100" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 41867 } } }
I20260808 18:25:03.375455 4770 sys_catalog.cc:458] T 00000000000000000000000000000000 P 874ed22651ea4961bc0120df29e12100 [sys.catalog]: This master's current role is: LEADER
I20260808 18:25:03.375716 4770 sys_catalog.cc:455] T 00000000000000000000000000000000 P 874ed22651ea4961bc0120df29e12100 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 874ed22651ea4961bc0120df29e12100. Latest consensus state: current_term: 1 leader_uuid: "874ed22651ea4961bc0120df29e12100" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "874ed22651ea4961bc0120df29e12100" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 41867 } } }
I20260808 18:25:03.375869 4770 sys_catalog.cc:458] T 00000000000000000000000000000000 P 874ed22651ea4961bc0120df29e12100 [sys.catalog]: This master's current role is: LEADER
I20260808 18:25:03.376173 4775 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:25:03.376937 4768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:03.360755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42976 (local address 127.23.72.190:41867)
0808 18:25:03.361245 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:03.361264 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:03.361313 (+ 49us) server_negotiation.cc:407] Connection header received
0808 18:25:03.362131 (+ 818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:03.362156 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:03.362599 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:03.362967 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:25:03.363954 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:03.365398 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:03.366647 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:03.367373 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:03.367568 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:03.370679 (+ 3111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:25:03.370685 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:25:03.370701 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:25:03.370732 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:25:03.372948 (+ 2216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:03.373630 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:03.373636 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:03.373641 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:03.373768 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:03.374083 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:03.374086 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:03.374088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:03.374574 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:25:03.374778 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:03.375577 (+ 799us) server_negotiation.cc:299] Negotiation successful
0808 18:25:03.375914 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":136,"threads_started":1}
I20260808 18:25:03.380764 4775 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:25:03.381211 4765 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 18:25:03.390595 4775 catalog_manager.cc:1347] Generated new cluster ID: 8707aa0c2f8748cf9d91b2efc2dd9443
I20260808 18:25:03.390669 4775 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:25:03.405700 4775 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:25:03.407061 4775 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:25:03.411571 4775 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 874ed22651ea4961bc0120df29e12100: Generated new TSK 0
I20260808 18:25:03.412416 4775 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.72.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab.
Entry for principal kudu/127.23.72.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab.
I20260808 18:25:04.122742 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.72.148:54221/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.129:0
--local_ip_for_outbound_sockets=127.23.72.129
--webserver_interface=127.23.72.129
--webserver_port=0
--tserver_master_addrs=127.23.72.190:41867
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab
--principal=kudu/127.23.72.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:40107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:25:04.290407 4812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:25:04.290774 4812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:25:04.290863 4812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:25:04.297096 4812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:25:04.297178 4812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:25:04.297284 4812 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.129
I20260808 18:25:04.303303 4812 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:40107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.23.72.148:54221/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab
--principal=kudu/127.23.72.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.72.129
--webserver_port=0
--tserver_master_addrs=127.23.72.190:41867
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4812
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.129
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:25:04.304452 4812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:25:04.305531 4812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:25:04.318063 4818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:04.318069 4819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:04.318441 4821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:25:04.319128 4812 server_base.cc:1034] running on GCE node
I20260808 18:25:04.319599 4812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:25:04.320326 4812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:25:04.321520 4812 hybrid_clock.cc:648] HybridClock initialized: now 1786213504321489 us; error 44 us; skew 500 ppm
Aug 08 18:25:04 dist-test-slave-5k54 krb5kdc[4215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213504, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:25:04.325011 4812 init.cc:377] Logged in from keytab as kudu/127.23.72.129@KRBTEST.COM (short username kudu)
I20260808 18:25:04.326188 4812 webserver.cc:460] Webserver started at http://127.23.72.129:37059/ using document root <none> and password file <none>
I20260808 18:25:04.326953 4812 fs_manager.cc:359] Metadata directory not provided
I20260808 18:25:04.327086 4812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:25:04.327381 4812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 18:25:04 dist-test-slave-5k54 krb5kdc[4215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213504, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for HTTP/127.23.72.148@KRBTEST.COM
I20260808 18:25:04.345985 4812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-0/data/instance:
uuid: "0c96af460dfa44a5a28964062a7ba43f"
format_stamp: "Formatted at 2026-08-08 18:25:04 on dist-test-slave-5k54"
server_key: "86cae428a752f6d96aee813f634bcefc"
server_key_iv: "bf5f6f9631b6ebfe8ccc37df5ddbb1f6"
server_key_version: "kuduclusterkey@0"
I20260808 18:25:04.346827 4812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance:
uuid: "0c96af460dfa44a5a28964062a7ba43f"
format_stamp: "Formatted at 2026-08-08 18:25:04 on dist-test-slave-5k54"
server_key: "86cae428a752f6d96aee813f634bcefc"
server_key_iv: "bf5f6f9631b6ebfe8ccc37df5ddbb1f6"
server_key_version: "kuduclusterkey@0"
I20260808 18:25:04.362581 4812 fs_manager.cc:691] Time spent creating directory manager: real 0.014s user 0.004s sys 0.000s
I20260808 18:25:04.384788 4831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:04.386528 4812 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 18:25:04.386654 4812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-0/wal
uuid: "0c96af460dfa44a5a28964062a7ba43f"
format_stamp: "Formatted at 2026-08-08 18:25:04 on dist-test-slave-5k54"
server_key: "86cae428a752f6d96aee813f634bcefc"
server_key_iv: "bf5f6f9631b6ebfe8ccc37df5ddbb1f6"
server_key_version: "kuduclusterkey@0"
I20260808 18:25:04.386790 4812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:25:04.428718 4812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:25:04.432608 4812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:25:04.432883 4812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:25:04.433729 4812 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:25:04.434900 4812 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:25:04.435024 4812 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:04.435158 4812 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:25:04.435251 4812 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:04.447505 4812 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.129:32961
I20260808 18:25:04.447610 4952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.129:32961 every 8 connection(s)
I20260808 18:25:04.448789 4812 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
I20260808 18:25:04.454016 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 4812
I20260808 18:25:04.454170 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance
I20260808 18:25:04.454504 23842 external_mini_cluster.cc:1428] Setting key ace0ce028d78dcf340c4ab154961e4d6
Aug 08 18:25:04 dist-test-slave-5k54 krb5kdc[4215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213504, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:25:04.468802 4955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:04.450927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:55333 (local address 127.23.72.190:41867)
0808 18:25:04.451190 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:04.451194 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:04.454177 (+ 2983us) server_negotiation.cc:407] Connection header received
0808 18:25:04.454295 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:04.454299 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:04.454383 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:04.454474 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:25:04.457495 (+ 3021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:04.458427 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:04.460273 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:04.460649 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:04.460762 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:04.464040 (+ 3278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:25:04.464044 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:25:04.464047 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:25:04.464075 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:25:04.466126 (+ 2051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:04.466834 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:04.466838 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:04.466840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:04.466932 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:04.467340 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:04.467344 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:04.467346 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:04.467603 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:25:04.467697 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:04.468434 (+ 737us) server_negotiation.cc:299] Negotiation successful
0808 18:25:04.468603 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":86,"threads_started":1}
I20260808 18:25:04.470757 4956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:04.451482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41867 (local address 127.23.72.129:55333)
0808 18:25:04.452653 (+ 1171us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:04.452712 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:04.453798 (+ 1086us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:04.454887 (+ 1089us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:04.456165 (+ 1278us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:04.456579 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:25:04.457289 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:04.457324 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:04.458624 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:04.458628 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:04.460130 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:04.460142 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:04.460956 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:04.460960 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:04.461156 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:04.461902 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:25:04.461926 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:25:04.463831 (+ 1905us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:25:04.466268 (+ 2437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:04.466277 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:04.466296 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:04.466699 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:04.467045 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:04.467048 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:04.467050 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:04.467221 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:04.467768 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:25:04.467775 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:25:04.468104 (+ 329us) client_negotiation.cc:769] Sending connection context
0808 18:25:04.469302 (+ 1198us) client_negotiation.cc:240] Negotiation successful
0808 18:25:04.469675 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":952,"thread_start_us":133,"threads_started":1}
I20260808 18:25:04.474728 4953 heartbeater.cc:347] Connected to a master server at 127.23.72.190:41867
I20260808 18:25:04.475154 4953 heartbeater.cc:464] Registering TS with master...
I20260808 18:25:04.475889 4953 heartbeater.cc:511] Master 127.23.72.190:41867 requested a full tablet report, sending...
I20260808 18:25:04.477802 4719 ts_manager.cc:194] Registered new tserver with Master: 0c96af460dfa44a5a28964062a7ba43f (127.23.72.129:32961)
I20260808 18:25:04.479452 4719 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.129@KRBTEST.COM'} at 127.23.72.129:55333
WARNING: no policy specified for kudu/127.23.72.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab.
Entry for principal kudu/127.23.72.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab.
I20260808 18:25:04.515586 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.72.148:54221/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.130:0
--local_ip_for_outbound_sockets=127.23.72.130
--webserver_interface=127.23.72.130
--webserver_port=0
--tserver_master_addrs=127.23.72.190:41867
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab
--principal=kudu/127.23.72.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:40107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 18:25:04 dist-test-slave-5k54 krb5kdc[4215](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786213504, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 18:25:04.676061 4959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:25:04.676502 4959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:25:04.676594 4959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:25:04.682746 4959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:25:04.682839 4959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:25:04.682927 4959 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.130
I20260808 18:25:04.691009 4959 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:40107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.23.72.148:54221/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab
--principal=kudu/127.23.72.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.72.130
--webserver_port=0
--tserver_master_addrs=127.23.72.190:41867
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4959
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.130
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:25:04.692610 4959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:25:04.694039 4959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:25:04.707605 4968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:04.707597 4966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:04.708122 4965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:25:04.712716 4959 server_base.cc:1034] running on GCE node
I20260808 18:25:04.713260 4959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:25:04.714236 4959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:25:04.715507 4959 hybrid_clock.cc:648] HybridClock initialized: now 1786213504715481 us; error 44 us; skew 500 ppm
Aug 08 18:25:04 dist-test-slave-5k54 krb5kdc[4215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213504, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:25:04.720474 4959 init.cc:377] Logged in from keytab as kudu/127.23.72.130@KRBTEST.COM (short username kudu)
I20260808 18:25:04.721614 4959 webserver.cc:460] Webserver started at http://127.23.72.130:38567/ using document root <none> and password file <none>
I20260808 18:25:04.722358 4959 fs_manager.cc:359] Metadata directory not provided
I20260808 18:25:04.722453 4959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:25:04.722685 4959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 18:25:04 dist-test-slave-5k54 krb5kdc[4215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213504, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.130@KRBTEST.COM for HTTP/127.23.72.148@KRBTEST.COM
I20260808 18:25:04.750532 4959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-1/data/instance:
uuid: "66ffdbc34d7e4c259b0d8c7938bdded9"
format_stamp: "Formatted at 2026-08-08 18:25:04 on dist-test-slave-5k54"
server_key: "17f39053482743fab499053b78017936"
server_key_iv: "34bf0cd67aa4c376dded883f94cedcf6"
server_key_version: "kuduclusterkey@0"
I20260808 18:25:04.751683 4959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance:
uuid: "66ffdbc34d7e4c259b0d8c7938bdded9"
format_stamp: "Formatted at 2026-08-08 18:25:04 on dist-test-slave-5k54"
server_key: "17f39053482743fab499053b78017936"
server_key_iv: "34bf0cd67aa4c376dded883f94cedcf6"
server_key_version: "kuduclusterkey@0"
I20260808 18:25:04.760982 4959 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.004s sys 0.000s
I20260808 18:25:04.779336 4975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:04.780887 4959 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.000s
I20260808 18:25:04.781054 4959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-1/wal
uuid: "66ffdbc34d7e4c259b0d8c7938bdded9"
format_stamp: "Formatted at 2026-08-08 18:25:04 on dist-test-slave-5k54"
server_key: "17f39053482743fab499053b78017936"
server_key_iv: "34bf0cd67aa4c376dded883f94cedcf6"
server_key_version: "kuduclusterkey@0"
I20260808 18:25:04.781221 4959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:25:04.801783 4959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:25:04.804976 4959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:25:04.805158 4959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:25:04.805904 4959 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:25:04.806849 4959 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:25:04.806931 4959 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:04.807062 4959 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:25:04.807266 4959 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:04.841868 4959 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.130:33909
I20260808 18:25:04.843384 4959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
I20260808 18:25:04.844129 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 4959
I20260808 18:25:04.844308 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance
I20260808 18:25:04.844780 23842 external_mini_cluster.cc:1428] Setting key 3dd9ba79620d69d09eb32f11522b531c
I20260808 18:25:04.850207 5093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.130:33909 every 8 connection(s)
WARNING: no policy specified for kudu/127.23.72.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.131@KRBTEST.COM" created.
Aug 08 18:25:04 dist-test-slave-5k54 krb5kdc[4215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213504, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.130@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:25:04.876304 4955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:04.855515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:36211 (local address 127.23.72.190:41867)
0808 18:25:04.855651 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:04.855656 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:04.859988 (+ 4332us) server_negotiation.cc:407] Connection header received
0808 18:25:04.861320 (+ 1332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:04.861325 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:04.861422 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:04.861498 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:25:04.863280 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:04.864160 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:04.865963 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:04.866521 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:04.866662 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:04.870788 (+ 4126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:25:04.870792 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:25:04.870795 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:25:04.870826 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:25:04.872687 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:04.873343 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:04.873347 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:04.873349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:04.873440 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:04.873856 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:04.873860 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:04.873861 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:04.874076 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:25:04.874167 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:04.875986 (+ 1819us) server_negotiation.cc:299] Negotiation successful
0808 18:25:04.876125 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 18:25:04.877322 5097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:04.854250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41867 (local address 127.23.72.130:36211)
0808 18:25:04.859811 (+ 5561us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:04.859873 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:04.861032 (+ 1159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:04.861960 (+ 928us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:04.861970 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:04.862356 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:25:04.863075 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:04.863097 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:04.864349 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:04.864353 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:04.865805 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:04.865816 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:04.866914 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:04.866921 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:04.867183 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:04.868256 (+ 1073us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:25:04.868286 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:25:04.870570 (+ 2284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:25:04.872831 (+ 2261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:04.872837 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:04.872854 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:04.873201 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:04.873550 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:04.873553 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:04.873555 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:04.873742 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:04.874222 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:25:04.874229 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:25:04.874555 (+ 326us) client_negotiation.cc:769] Sending connection context
0808 18:25:04.874766 (+ 211us) client_negotiation.cc:240] Negotiation successful
0808 18:25:04.875104 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5293,"thread_start_us":225,"threads_started":1}
I20260808 18:25:04.878984 5095 heartbeater.cc:347] Connected to a master server at 127.23.72.190:41867
I20260808 18:25:04.879248 5095 heartbeater.cc:464] Registering TS with master...
I20260808 18:25:04.879879 5095 heartbeater.cc:511] Master 127.23.72.190:41867 requested a full tablet report, sending...
I20260808 18:25:04.881130 4719 ts_manager.cc:194] Registered new tserver with Master: 66ffdbc34d7e4c259b0d8c7938bdded9 (127.23.72.130:33909)
I20260808 18:25:04.881930 4719 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.130@KRBTEST.COM'} at 127.23.72.130:36211
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab.
Entry for principal kudu/127.23.72.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab.
I20260808 18:25:04.898931 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.72.148:54221/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.131:0
--local_ip_for_outbound_sockets=127.23.72.131
--webserver_interface=127.23.72.131
--webserver_port=0
--tserver_master_addrs=127.23.72.190:41867
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab
--principal=kudu/127.23.72.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:40107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 18:25:04 dist-test-slave-5k54 krb5kdc[4215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213504, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for HTTP/127.23.72.148@KRBTEST.COM
W20260808 18:25:05.108740 5099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:25:05.109136 5099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:25:05.109200 5099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:25:05.115197 5099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:25:05.115284 5099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:25:05.115499 5099 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.131
I20260808 18:25:05.123673 5099 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:40107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.23.72.148:54221/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab
--principal=kudu/127.23.72.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.72.131
--webserver_port=0
--tserver_master_addrs=127.23.72.190:41867
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5099
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.131
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:25:05.125104 5099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:25:05.126448 5099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:25:05.139550 5105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:05.139748 5108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:05.139550 5104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:25:05.140933 5099 server_base.cc:1034] running on GCE node
I20260808 18:25:05.141346 5099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:25:05.142010 5099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:25:05.143199 5099 hybrid_clock.cc:648] HybridClock initialized: now 1786213505143170 us; error 41 us; skew 500 ppm
Aug 08 18:25:05 dist-test-slave-5k54 krb5kdc[4215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213505, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:25:05.146497 5099 init.cc:377] Logged in from keytab as kudu/127.23.72.131@KRBTEST.COM (short username kudu)
I20260808 18:25:05.147527 5099 webserver.cc:460] Webserver started at http://127.23.72.131:35443/ using document root <none> and password file <none>
I20260808 18:25:05.148082 5099 fs_manager.cc:359] Metadata directory not provided
I20260808 18:25:05.148171 5099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:25:05.148427 5099 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 18:25:05 dist-test-slave-5k54 krb5kdc[4215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213505, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.131@KRBTEST.COM for HTTP/127.23.72.148@KRBTEST.COM
I20260808 18:25:05.163331 5099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-2/data/instance:
uuid: "50cfb2e821534ee5b59c14854f08d212"
format_stamp: "Formatted at 2026-08-08 18:25:05 on dist-test-slave-5k54"
server_key: "66a05f43c28eac368e6165031c8ffaa2"
server_key_iv: "56b01d398005b586cc7944c7f5f2f997"
server_key_version: "kuduclusterkey@0"
I20260808 18:25:05.172827 5099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance:
uuid: "50cfb2e821534ee5b59c14854f08d212"
format_stamp: "Formatted at 2026-08-08 18:25:05 on dist-test-slave-5k54"
server_key: "66a05f43c28eac368e6165031c8ffaa2"
server_key_iv: "56b01d398005b586cc7944c7f5f2f997"
server_key_version: "kuduclusterkey@0"
I20260808 18:25:05.181031 5099 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.004s sys 0.000s
I20260808 18:25:05.199852 5116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:05.201661 5099 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.000s
I20260808 18:25:05.201784 5099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-2/wal
uuid: "50cfb2e821534ee5b59c14854f08d212"
format_stamp: "Formatted at 2026-08-08 18:25:05 on dist-test-slave-5k54"
server_key: "66a05f43c28eac368e6165031c8ffaa2"
server_key_iv: "56b01d398005b586cc7944c7f5f2f997"
server_key_version: "kuduclusterkey@0"
I20260808 18:25:05.201906 5099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:25:05.219406 5099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:25:05.222513 5099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:25:05.222715 5099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:25:05.223352 5099 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:25:05.224287 5099 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:25:05.224363 5099 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:05.224463 5099 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:25:05.224522 5099 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:05.244042 5099 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.131:35105
I20260808 18:25:05.244112 5238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.131:35105 every 8 connection(s)
I20260808 18:25:05.245136 5099 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
I20260808 18:25:05.247150 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 5099
I20260808 18:25:05.247294 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance
I20260808 18:25:05.247812 23842 external_mini_cluster.cc:1428] Setting key 4c8a7569e8a4861ca44b4f2936a5d088
Aug 08 18:25:05 dist-test-slave-5k54 krb5kdc[4215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213505, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.131@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:25:05.263651 4955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:05.249575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:41175 (local address 127.23.72.190:41867)
0808 18:25:05.249696 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:05.249701 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:05.250343 (+ 642us) server_negotiation.cc:407] Connection header received
0808 18:25:05.251531 (+ 1188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:05.251534 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:05.251602 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:05.251696 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:25:05.253579 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:05.254219 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:05.255475 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:05.255716 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:05.255846 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:05.259112 (+ 3266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:25:05.259118 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:25:05.259121 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:25:05.259153 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:25:05.260895 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:05.261540 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:05.261543 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:05.261545 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:05.261645 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:05.262323 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:05.262327 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:05.262328 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:05.262571 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:25:05.262645 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:05.263307 (+ 662us) server_negotiation.cc:299] Negotiation successful
0808 18:25:05.263481 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 18:25:05.264726 5242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:05.248354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41867 (local address 127.23.72.131:41175)
0808 18:25:05.250159 (+ 1805us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:05.250219 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:05.251278 (+ 1059us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:05.252020 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:05.252029 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:05.252564 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:25:05.253383 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:05.253407 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:05.254365 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:05.254367 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:05.255322 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:05.255331 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:05.255878 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:05.255882 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:05.256111 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:05.256846 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:25:05.256868 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:25:05.258920 (+ 2052us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:25:05.261033 (+ 2113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:05.261040 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:05.261057 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:05.261422 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:05.261998 (+ 576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:05.262004 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:05.262009 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:05.262212 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:05.262776 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:25:05.262781 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:25:05.263057 (+ 276us) client_negotiation.cc:769] Sending connection context
0808 18:25:05.263264 (+ 207us) client_negotiation.cc:240] Negotiation successful
0808 18:25:05.263615 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1532,"thread_start_us":210,"threads_started":1}
I20260808 18:25:05.266026 5239 heartbeater.cc:347] Connected to a master server at 127.23.72.190:41867
I20260808 18:25:05.266285 5239 heartbeater.cc:464] Registering TS with master...
I20260808 18:25:05.266793 5239 heartbeater.cc:511] Master 127.23.72.190:41867 requested a full tablet report, sending...
I20260808 18:25:05.267925 4719 ts_manager.cc:194] Registered new tserver with Master: 50cfb2e821534ee5b59c14854f08d212 (127.23.72.131:35105)
I20260808 18:25:05.268514 4719 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.131@KRBTEST.COM'} at 127.23.72.131:41175
I20260808 18:25:05.274619 23842 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:25:05.275475 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 4812
I20260808 18:25:05.294832 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 4959
I20260808 18:25:05.304239 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 5099
I20260808 18:25:05.315655 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 4692
I20260808 18:25:05.324580 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:41867
--webserver_interface=127.23.72.190
--webserver_port=42279
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:40107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:41867
--ranger_config_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.72.148:54221/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.23.72.148:54221/kms with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:25:05.439602 5243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:25:05.439885 5243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:25:05.439978 5243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:25:05.444089 5243 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:25:05.444150 5243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:25:05.444170 5243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:25:05.444187 5243 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:25:05.444250 5243 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:25:05.449923 5243 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:40107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.23.72.148:54221/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:41867
--ranger_config_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:41867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.72.190
--webserver_port=42279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:25:05.450996 5243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:25:05.452071 5243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:25:05.460764 5253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:05.460786 5249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:25:05.460898 5243 server_base.cc:1034] running on GCE node
W20260808 18:25:05.460773 5250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:25:05.461608 5243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:25:05.462657 5243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:25:05.463819 5243 hybrid_clock.cc:648] HybridClock initialized: now 1786213505463798 us; error 34 us; skew 500 ppm
Aug 08 18:25:05 dist-test-slave-5k54 krb5kdc[4215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213505, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:25:05.466563 5243 init.cc:377] Logged in from keytab as kudu/127.23.72.190@KRBTEST.COM (short username kudu)
I20260808 18:25:05.467507 5243 webserver.cc:460] Webserver started at http://127.23.72.190:42279/ using document root <none> and password file <none>
I20260808 18:25:05.468055 5243 fs_manager.cc:359] Metadata directory not provided
I20260808 18:25:05.468158 5243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 18:25:05 dist-test-slave-5k54 krb5kdc[4215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213505, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for HTTP/127.23.72.148@KRBTEST.COM
I20260808 18:25:05.515611 5243 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 18:25:05.518442 5261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:05.520202 5243 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:25:05.520360 5243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "874ed22651ea4961bc0120df29e12100"
format_stamp: "Formatted at 2026-08-08 18:25:03 on dist-test-slave-5k54"
server_key: "93b88b5222175c73dfca4828613c5053"
server_key_iv: "95fab39dbcdbd1492941169d55f5fcad"
server_key_version: "kuduclusterkey@0"
I20260808 18:25:05.521081 5243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:25:05.555172 5243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:25:05.558257 5243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:25:05.558485 5243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:25:05.566980 5243 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.190:41867
I20260808 18:25:05.566988 5316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.190:41867 every 8 connection(s)
I20260808 18:25:05.568114 5243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:25:05.573235 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 5243
I20260808 18:25:05.573737 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.72.148:54221/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.129:32961
--local_ip_for_outbound_sockets=127.23.72.129
--tserver_master_addrs=127.23.72.190:41867
--webserver_port=37059
--webserver_interface=127.23.72.129
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab
--principal=kudu/127.23.72.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:40107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.23.72.148:54221/kms with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:25:05.573789 5317 sys_catalog.cc:263] Verifying existing consensus state
I20260808 18:25:05.576048 5317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 874ed22651ea4961bc0120df29e12100: Bootstrap starting.
I20260808 18:25:05.598623 5317 log.cc:824] T 00000000000000000000000000000000 P 874ed22651ea4961bc0120df29e12100: Log is configured to *not* fsync() on all Append() calls
I20260808 18:25:05.607182 5317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 874ed22651ea4961bc0120df29e12100: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 18:25:05.607687 5317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 874ed22651ea4961bc0120df29e12100: Bootstrap complete.
I20260808 18:25:05.613538 5317 raft_consensus.cc:348] T 00000000000000000000000000000000 P 874ed22651ea4961bc0120df29e12100 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "874ed22651ea4961bc0120df29e12100" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 41867 } }
I20260808 18:25:05.614058 5317 raft_consensus.cc:729] T 00000000000000000000000000000000 P 874ed22651ea4961bc0120df29e12100 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 874ed22651ea4961bc0120df29e12100, State: Initialized, Role: FOLLOWER
I20260808 18:25:05.614648 5317 consensus_queue.cc:260] T 00000000000000000000000000000000 P 874ed22651ea4961bc0120df29e12100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "874ed22651ea4961bc0120df29e12100" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 41867 } }
I20260808 18:25:05.614804 5317 raft_consensus.cc:388] T 00000000000000000000000000000000 P 874ed22651ea4961bc0120df29e12100 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:25:05.614912 5317 raft_consensus.cc:482] T 00000000000000000000000000000000 P 874ed22651ea4961bc0120df29e12100 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:25:05.615013 5317 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 874ed22651ea4961bc0120df29e12100 [term 1 FOLLOWER]: Advancing to term 2
I20260808 18:25:05.617558 5317 raft_consensus.cc:504] T 00000000000000000000000000000000 P 874ed22651ea4961bc0120df29e12100 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "874ed22651ea4961bc0120df29e12100" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 41867 } }
I20260808 18:25:05.617951 5317 leader_election.cc:302] T 00000000000000000000000000000000 P 874ed22651ea4961bc0120df29e12100 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 874ed22651ea4961bc0120df29e12100; no voters:
I20260808 18:25:05.618373 5317 leader_election.cc:288] T 00000000000000000000000000000000 P 874ed22651ea4961bc0120df29e12100 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 18:25:05.618513 5323 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 874ed22651ea4961bc0120df29e12100 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 18:25:05.618920 5323 raft_consensus.cc:686] T 00000000000000000000000000000000 P 874ed22651ea4961bc0120df29e12100 [term 2 LEADER]: Becoming Leader. State: Replica: 874ed22651ea4961bc0120df29e12100, State: Running, Role: LEADER
I20260808 18:25:05.619274 5323 consensus_queue.cc:237] T 00000000000000000000000000000000 P 874ed22651ea4961bc0120df29e12100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "874ed22651ea4961bc0120df29e12100" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 41867 } }
I20260808 18:25:05.619753 5317 sys_catalog.cc:564] T 00000000000000000000000000000000 P 874ed22651ea4961bc0120df29e12100 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:25:05.620326 5323 sys_catalog.cc:455] T 00000000000000000000000000000000 P 874ed22651ea4961bc0120df29e12100 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "874ed22651ea4961bc0120df29e12100" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "874ed22651ea4961bc0120df29e12100" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 41867 } } }
I20260808 18:25:05.620510 5323 sys_catalog.cc:458] T 00000000000000000000000000000000 P 874ed22651ea4961bc0120df29e12100 [sys.catalog]: This master's current role is: LEADER
I20260808 18:25:05.620767 5323 sys_catalog.cc:455] T 00000000000000000000000000000000 P 874ed22651ea4961bc0120df29e12100 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 874ed22651ea4961bc0120df29e12100. Latest consensus state: current_term: 2 leader_uuid: "874ed22651ea4961bc0120df29e12100" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "874ed22651ea4961bc0120df29e12100" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 41867 } } }
I20260808 18:25:05.620906 5323 sys_catalog.cc:458] T 00000000000000000000000000000000 P 874ed22651ea4961bc0120df29e12100 [sys.catalog]: This master's current role is: LEADER
I20260808 18:25:05.621192 5330 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:25:05.624315 5330 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:25:05.624877 5317 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 18:25:05.627321 5330 catalog_manager.cc:1259] Loaded cluster ID: 8707aa0c2f8748cf9d91b2efc2dd9443
I20260808 18:25:05.627554 5330 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:25:05.630072 5330 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:25:05.630885 5330 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 874ed22651ea4961bc0120df29e12100: Loaded TSK: 0
I20260808 18:25:05.632211 5330 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 18:25:05.721697 5320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:25:05.722234 5320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:25:05.722388 5320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:25:05.729552 5320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:25:05.729703 5320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:25:05.729864 5320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.129
I20260808 18:25:05.738348 5320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:40107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.23.72.148:54221/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab
--principal=kudu/127.23.72.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.129:32961
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.72.129
--webserver_port=37059
--tserver_master_addrs=127.23.72.190:41867
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.129
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:25:05.740022 5320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:25:05.741407 5320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:25:05.755020 5364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:05.755389 5362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:05.755626 5360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:25:05.755841 5320 server_base.cc:1034] running on GCE node
I20260808 18:25:05.756409 5320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:25:05.757135 5320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:25:05.758288 5320 hybrid_clock.cc:648] HybridClock initialized: now 1786213505758221 us; error 82 us; skew 500 ppm
Aug 08 18:25:05 dist-test-slave-5k54 krb5kdc[4215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213505, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:25:05.762542 5320 init.cc:377] Logged in from keytab as kudu/127.23.72.129@KRBTEST.COM (short username kudu)
I20260808 18:25:05.763649 5320 webserver.cc:460] Webserver started at http://127.23.72.129:37059/ using document root <none> and password file <none>
I20260808 18:25:05.764683 5320 fs_manager.cc:359] Metadata directory not provided
I20260808 18:25:05.764909 5320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 18:25:05 dist-test-slave-5k54 krb5kdc[4215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213505, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for HTTP/127.23.72.148@KRBTEST.COM
I20260808 18:25:05.787222 5320 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.000s
I20260808 18:25:05.792220 5373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:05.793910 5320 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260808 18:25:05.794037 5320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-0/wal
uuid: "0c96af460dfa44a5a28964062a7ba43f"
format_stamp: "Formatted at 2026-08-08 18:25:04 on dist-test-slave-5k54"
server_key: "86cae428a752f6d96aee813f634bcefc"
server_key_iv: "bf5f6f9631b6ebfe8ccc37df5ddbb1f6"
server_key_version: "kuduclusterkey@0"
I20260808 18:25:05.794760 5320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:25:05.842489 5320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:25:05.846340 5320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:25:05.846547 5320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:25:05.847234 5320 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:25:05.848356 5320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:25:05.848421 5320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:05.848481 5320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:25:05.848562 5320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260808 18:25:05.860049 5320 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.129:32961
I20260808 18:25:05.860148 5490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.129:32961 every 8 connection(s)
I20260808 18:25:05.861425 5320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
I20260808 18:25:05.861625 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 5320
I20260808 18:25:05.862102 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.72.148:54221/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.130:33909
--local_ip_for_outbound_sockets=127.23.72.130
--tserver_master_addrs=127.23.72.190:41867
--webserver_port=38567
--webserver_interface=127.23.72.130
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab
--principal=kudu/127.23.72.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:40107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.23.72.148:54221/kms with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 18:25:05 dist-test-slave-5k54 krb5kdc[4215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213505, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:25:05.892287 5495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:05.873532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41867 (local address 127.23.72.129:37069)
0808 18:25:05.874088 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:05.874142 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:05.875161 (+ 1019us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:05.876912 (+ 1751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:05.876920 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:05.877274 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:25:05.877976 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:05.877997 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:05.880306 (+ 2309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:05.880309 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:05.881747 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:05.881759 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:05.882843 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:05.882848 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:05.883043 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:05.883822 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:25:05.883844 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:25:05.886080 (+ 2236us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:25:05.888624 (+ 2544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:05.888630 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:05.888645 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:05.888988 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:05.889422 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:05.889425 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:05.889427 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:05.889605 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:05.890373 (+ 768us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:25:05.890380 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:25:05.890680 (+ 300us) client_negotiation.cc:769] Sending connection context
0808 18:25:05.890894 (+ 214us) client_negotiation.cc:240] Negotiation successful
0808 18:25:05.891228 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":139,"threads_started":1}
I20260808 18:25:05.894932 5494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:05.873363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:37069 (local address 127.23.72.190:41867)
0808 18:25:05.875692 (+ 2329us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:05.875715 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:05.875759 (+ 44us) server_negotiation.cc:407] Connection header received
0808 18:25:05.876055 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:05.876068 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:05.876458 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:05.878221 (+ 1763us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:25:05.878549 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:05.880150 (+ 1601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:05.881917 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:05.882641 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:05.883458 (+ 817us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:05.886300 (+ 2842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:25:05.886307 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:25:05.886322 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:25:05.886354 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:25:05.888487 (+ 2133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:05.889152 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:05.889157 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:05.889162 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:05.889304 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:05.889731 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:05.889734 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:05.889736 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:05.890231 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:25:05.893520 (+ 3289us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:05.893706 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 18:25:05.894023 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2160,"thread_start_us":131,"threads_started":1}
I20260808 18:25:05.897696 5491 heartbeater.cc:347] Connected to a master server at 127.23.72.190:41867
I20260808 18:25:05.897950 5491 heartbeater.cc:464] Registering TS with master...
I20260808 18:25:05.898524 5491 heartbeater.cc:511] Master 127.23.72.190:41867 requested a full tablet report, sending...
W20260808 18:25:05.900895 5491 heartbeater.cc:644] Failed to heartbeat to 127.23.72.190:41867 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 18:25:05.901019 5491 heartbeater.cc:464] Registering TS with master...
I20260808 18:25:05.901204 5491 heartbeater.cc:511] Master 127.23.72.190:41867 requested a full tablet report, sending...
I20260808 18:25:05.902210 5491 heartbeater.cc:464] Registering TS with master...
I20260808 18:25:05.902387 5491 heartbeater.cc:511] Master 127.23.72.190:41867 requested a full tablet report, sending...
W20260808 18:25:05.903235 5491 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 18:25:06.024312 5493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:25:06.024677 5493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:25:06.024749 5493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:25:06.031203 5493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:25:06.031561 5493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:25:06.031682 5493 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.130
I20260808 18:25:06.039162 5493 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:40107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.23.72.148:54221/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab
--principal=kudu/127.23.72.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.130:33909
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.72.130
--webserver_port=38567
--tserver_master_addrs=127.23.72.190:41867
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5493
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.130
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:25:06.040704 5493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:25:06.042042 5493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:25:06.054929 5501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:06.055498 5504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:06.055596 5500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:25:06.056504 5493 server_base.cc:1034] running on GCE node
I20260808 18:25:06.056991 5493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:25:06.057780 5493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:25:06.058984 5493 hybrid_clock.cc:648] HybridClock initialized: now 1786213506058963 us; error 42 us; skew 500 ppm
Aug 08 18:25:06 dist-test-slave-5k54 krb5kdc[4215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213506, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:25:06.062197 5493 init.cc:377] Logged in from keytab as kudu/127.23.72.130@KRBTEST.COM (short username kudu)
I20260808 18:25:06.063284 5493 webserver.cc:460] Webserver started at http://127.23.72.130:38567/ using document root <none> and password file <none>
I20260808 18:25:06.064010 5493 fs_manager.cc:359] Metadata directory not provided
I20260808 18:25:06.064117 5493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 18:25:06 dist-test-slave-5k54 krb5kdc[4215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213506, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.130@KRBTEST.COM for HTTP/127.23.72.148@KRBTEST.COM
I20260808 18:25:06.091655 5493 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 18:25:06.095608 5512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:06.097047 5493 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260808 18:25:06.097241 5493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-1/wal
uuid: "66ffdbc34d7e4c259b0d8c7938bdded9"
format_stamp: "Formatted at 2026-08-08 18:25:04 on dist-test-slave-5k54"
server_key: "17f39053482743fab499053b78017936"
server_key_iv: "34bf0cd67aa4c376dded883f94cedcf6"
server_key_version: "kuduclusterkey@0"
I20260808 18:25:06.097952 5493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:25:06.117743 5493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:25:06.120950 5493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:25:06.121167 5493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:25:06.121855 5493 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:25:06.122675 5493 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:25:06.122758 5493 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:06.122856 5493 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:25:06.122897 5493 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:06.134119 5493 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.130:33909
I20260808 18:25:06.134159 5635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.130:33909 every 8 connection(s)
I20260808 18:25:06.135298 5493 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
I20260808 18:25:06.136256 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 5493
I20260808 18:25:06.136783 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.72.148:54221/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.131:35105
--local_ip_for_outbound_sockets=127.23.72.131
--tserver_master_addrs=127.23.72.190:41867
--webserver_port=35443
--webserver_interface=127.23.72.131
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab
--principal=kudu/127.23.72.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:40107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.23.72.148:54221/kms with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 18:25:06 dist-test-slave-5k54 krb5kdc[4215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213506, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.130@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:25:06.152310 5494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:06.139607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:39151 (local address 127.23.72.190:41867)
0808 18:25:06.139783 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:06.139799 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:06.140233 (+ 434us) server_negotiation.cc:407] Connection header received
0808 18:25:06.141149 (+ 916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:06.141152 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:06.141212 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:06.141322 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:25:06.142476 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:06.143006 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:06.144471 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:06.144720 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:06.144802 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:06.148381 (+ 3579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:25:06.148385 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:25:06.148388 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:25:06.148421 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:25:06.150088 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:06.150605 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:06.150608 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:06.150610 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:06.150690 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:06.151042 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:06.151045 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:06.151047 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:06.151224 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:25:06.151336 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:06.151999 (+ 663us) server_negotiation.cc:299] Negotiation successful
0808 18:25:06.152143 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 18:25:06.154320 5639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:06.138420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41867 (local address 127.23.72.130:39151)
0808 18:25:06.140077 (+ 1657us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:06.140118 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:06.140943 (+ 825us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:06.141505 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:06.141512 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:06.141775 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:25:06.142317 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:06.142339 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:06.143153 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:06.143156 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:06.144310 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:06.144321 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:06.144965 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:06.144967 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:06.145104 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:06.145738 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:25:06.145754 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:25:06.148143 (+ 2389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:25:06.150208 (+ 2065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:06.150213 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:06.150227 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:06.150492 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:06.150785 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:06.150789 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:06.150791 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:06.150936 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:06.151341 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:25:06.151350 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:25:06.151702 (+ 352us) client_negotiation.cc:769] Sending connection context
0808 18:25:06.152875 (+ 1173us) client_negotiation.cc:240] Negotiation successful
0808 18:25:06.153210 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1468,"thread_start_us":142,"threads_started":1}
I20260808 18:25:06.155913 5636 heartbeater.cc:347] Connected to a master server at 127.23.72.190:41867
I20260808 18:25:06.156157 5636 heartbeater.cc:464] Registering TS with master...
I20260808 18:25:06.156713 5636 heartbeater.cc:511] Master 127.23.72.190:41867 requested a full tablet report, sending...
W20260808 18:25:06.158314 5636 heartbeater.cc:644] Failed to heartbeat to 127.23.72.190:41867 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 18:25:06.158439 5636 heartbeater.cc:464] Registering TS with master...
I20260808 18:25:06.158583 5636 heartbeater.cc:511] Master 127.23.72.190:41867 requested a full tablet report, sending...
I20260808 18:25:06.159394 5636 heartbeater.cc:464] Registering TS with master...
I20260808 18:25:06.159572 5636 heartbeater.cc:511] Master 127.23.72.190:41867 requested a full tablet report, sending...
W20260808 18:25:06.160387 5636 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 18:25:06.285969 5638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:25:06.286269 5638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:25:06.286404 5638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:25:06.291675 5638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:25:06.291805 5638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:25:06.291916 5638 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.131
I20260808 18:25:06.299639 5638 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:40107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.23.72.148:54221/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab
--principal=kudu/127.23.72.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.131:35105
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.72.131
--webserver_port=35443
--tserver_master_addrs=127.23.72.190:41867
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5638
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.131
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:25:06.300805 5638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:25:06.302073 5638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:25:06.311980 5649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:06.312016 5645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:06.311996 5647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:25:06.312830 5638 server_base.cc:1034] running on GCE node
I20260808 18:25:06.313320 5638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:25:06.314031 5638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:25:06.315229 5638 hybrid_clock.cc:648] HybridClock initialized: now 1786213506315159 us; error 87 us; skew 500 ppm
Aug 08 18:25:06 dist-test-slave-5k54 krb5kdc[4215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213506, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:25:06.318943 5638 init.cc:377] Logged in from keytab as kudu/127.23.72.131@KRBTEST.COM (short username kudu)
I20260808 18:25:06.320075 5638 webserver.cc:460] Webserver started at http://127.23.72.131:35443/ using document root <none> and password file <none>
I20260808 18:25:06.320874 5638 fs_manager.cc:359] Metadata directory not provided
I20260808 18:25:06.321017 5638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 18:25:06 dist-test-slave-5k54 krb5kdc[4215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213506, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.131@KRBTEST.COM for HTTP/127.23.72.148@KRBTEST.COM
I20260808 18:25:06.343722 5638 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 18:25:06.347122 5658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:06.349166 5638 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260808 18:25:06.349296 5638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-2/wal
uuid: "50cfb2e821534ee5b59c14854f08d212"
format_stamp: "Formatted at 2026-08-08 18:25:05 on dist-test-slave-5k54"
server_key: "66a05f43c28eac368e6165031c8ffaa2"
server_key_iv: "56b01d398005b586cc7944c7f5f2f997"
server_key_version: "kuduclusterkey@0"
I20260808 18:25:06.350059 5638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:25:06.372148 5638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:25:06.375852 5638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:25:06.376068 5638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:25:06.376806 5638 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:25:06.377976 5638 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:25:06.378088 5638 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:06.378192 5638 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:25:06.378261 5638 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:06.389604 5638 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.131:35105
I20260808 18:25:06.389741 5777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.131:35105 every 8 connection(s)
I20260808 18:25:06.391000 5638 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
I20260808 18:25:06.400661 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 5638
Aug 08 18:25:06 dist-test-slave-5k54 krb5kdc[4215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213506, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.131@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:25:06.408250 5494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:06.393142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:59695 (local address 127.23.72.190:41867)
0808 18:25:06.393283 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:06.393288 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:06.394374 (+ 1086us) server_negotiation.cc:407] Connection header received
0808 18:25:06.395504 (+ 1130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:06.395508 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:06.395570 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:06.395672 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:25:06.396725 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:06.397555 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:06.399268 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:06.399626 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:06.399711 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:06.404069 (+ 4358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:25:06.404072 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:25:06.404075 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:25:06.404102 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:25:06.405877 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:06.406573 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:06.406577 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:06.406578 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:06.406665 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:06.407044 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:06.407047 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:06.407049 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:06.407286 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:25:06.407409 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:06.407961 (+ 552us) server_negotiation.cc:299] Negotiation successful
0808 18:25:06.408081 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 18:25:06.409139 5780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:06.393638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41867 (local address 127.23.72.131:59695)
0808 18:25:06.394209 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:06.394265 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:06.395245 (+ 980us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:06.395821 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:06.395839 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:06.396106 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:25:06.396563 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:06.396576 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:06.397685 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:06.397688 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:06.399071 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:06.399083 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:06.399947 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:06.399951 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:06.400219 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:06.400895 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:25:06.400916 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:25:06.403876 (+ 2960us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:25:06.406001 (+ 2125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:06.406007 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:06.406023 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:06.406459 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:06.406772 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:06.406775 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:06.406778 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:06.406937 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:06.407418 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:25:06.407452 (+ 34us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:25:06.407719 (+ 267us) client_negotiation.cc:769] Sending connection context
0808 18:25:06.407920 (+ 201us) client_negotiation.cc:240] Negotiation successful
0808 18:25:06.408230 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":141,"threads_started":1}
I20260808 18:25:06.410683 5778 heartbeater.cc:347] Connected to a master server at 127.23.72.190:41867
I20260808 18:25:06.410929 5778 heartbeater.cc:464] Registering TS with master...
I20260808 18:25:06.411494 5778 heartbeater.cc:511] Master 127.23.72.190:41867 requested a full tablet report, sending...
W20260808 18:25:06.413153 5778 heartbeater.cc:644] Failed to heartbeat to 127.23.72.190:41867 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 18:25:06.413302 5778 heartbeater.cc:464] Registering TS with master...
I20260808 18:25:06.413481 5778 heartbeater.cc:511] Master 127.23.72.190:41867 requested a full tablet report, sending...
I20260808 18:25:06.414294 5778 heartbeater.cc:464] Registering TS with master...
I20260808 18:25:06.414467 5778 heartbeater.cc:511] Master 127.23.72.190:41867 requested a full tablet report, sending...
W20260808 18:25:06.415308 5778 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 18:25:06.415546 5782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:06.401592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40102 (local address 127.23.72.190:41867)
0808 18:25:06.402401 (+ 809us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:06.402413 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:06.402425 (+ 12us) server_negotiation.cc:407] Connection header received
0808 18:25:06.402465 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:06.402468 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:06.402504 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:06.402602 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:25:06.403528 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:06.404336 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:06.405997 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:06.406338 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:06.406443 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:06.409569 (+ 3126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:25:06.409576 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:25:06.409580 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:25:06.409604 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:25:06.412737 (+ 3133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:06.413695 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:06.413698 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:06.413699 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:06.413771 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:06.414367 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:06.414370 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:06.414371 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:06.414579 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:25:06.414913 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:06.415186 (+ 273us) server_negotiation.cc:299] Negotiation successful
0808 18:25:06.415317 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":695,"thread_start_us":101,"threads_started":1}
Aug 08 18:25:06 dist-test-slave-5k54 krb5kdc[4215](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786213506, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:25:06.903539 5491 heartbeater.cc:464] Registering TS with master...
I20260808 18:25:06.903887 5491 heartbeater.cc:511] Master 127.23.72.190:41867 requested a full tablet report, sending...
I20260808 18:25:06.905023 5279 ts_manager.cc:194] Registered new tserver with Master: 0c96af460dfa44a5a28964062a7ba43f (127.23.72.129:32961)
I20260808 18:25:06.906272 5279 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.129@KRBTEST.COM'} at 127.23.72.129:37069
I20260808 18:25:07.160636 5636 heartbeater.cc:464] Registering TS with master...
I20260808 18:25:07.161020 5636 heartbeater.cc:511] Master 127.23.72.190:41867 requested a full tablet report, sending...
I20260808 18:25:07.163067 5279 ts_manager.cc:194] Registered new tserver with Master: 66ffdbc34d7e4c259b0d8c7938bdded9 (127.23.72.130:33909)
I20260808 18:25:07.163902 5279 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.130@KRBTEST.COM'} at 127.23.72.130:39151
Aug 08 18:25:07 dist-test-slave-5k54 krb5kdc[4215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213506, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for HTTP/127.23.72.148@KRBTEST.COM
I20260808 18:25:07.415539 5778 heartbeater.cc:464] Registering TS with master...
I20260808 18:25:07.416062 5778 heartbeater.cc:511] Master 127.23.72.190:41867 requested a full tablet report, sending...
I20260808 18:25:07.416862 5279 ts_manager.cc:194] Registered new tserver with Master: 50cfb2e821534ee5b59c14854f08d212 (127.23.72.131:35105)
I20260808 18:25:07.417621 5279 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.131@KRBTEST.COM'} at 127.23.72.131:59695
I20260808 18:25:07.427042 23842 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:25:07.440375 5802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:07.429141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40116 (local address 127.23.72.190:41867)
0808 18:25:07.429461 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:07.429466 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:07.429975 (+ 509us) server_negotiation.cc:407] Connection header received
0808 18:25:07.430146 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:07.430158 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:07.430209 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:07.430299 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:25:07.431271 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:07.432155 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:07.433449 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:07.433840 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:07.433962 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:07.436163 (+ 2201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:25:07.436167 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:25:07.436171 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:25:07.436202 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:25:07.438280 (+ 2078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:07.438816 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:07.438820 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:07.438822 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:07.438934 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:07.439266 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:07.439269 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:07.439271 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:07.439547 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:25:07.439718 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:07.439995 (+ 277us) server_negotiation.cc:299] Negotiation successful
0808 18:25:07.440146 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":119,"threads_started":1}
I20260808 18:25:07.445510 5279 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40116:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:25:07.451013 5279 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:25:07.475512 5809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:07.468954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.131:35105 (local address 127.0.0.1:52078)
0808 18:25:07.469848 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:07.469862 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:07.469963 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:07.470626 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:07.470629 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:07.470647 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:25:07.470791 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:07.470798 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:07.472460 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:07.472463 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:07.473794 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:07.473805 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:07.474980 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:07.474984 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:07.475087 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:07.475128 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 18:25:07.475205 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 18:25:07.475309 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":826,"mutex_wait_us":27,"thread_start_us":149,"threads_started":1}
I20260808 18:25:07.476701 5813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:07.469434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52078 (local address 127.23.72.131:35105)
0808 18:25:07.470182 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:07.470189 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:07.470209 (+ 20us) server_negotiation.cc:407] Connection header received
0808 18:25:07.470279 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:07.470284 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:07.470476 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:07.470940 (+ 464us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:25:07.471170 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:07.472328 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:07.473986 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:07.474828 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:07.476248 (+ 1420us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:07.476405 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:07.476453 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 18:25:07.476536 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":649,"thread_start_us":115,"threads_started":1}
I20260808 18:25:07.477210 5811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:07.468208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41804 (local address 127.23.72.129:32961)
0808 18:25:07.468662 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:07.468669 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:07.469718 (+ 1049us) server_negotiation.cc:407] Connection header received
0808 18:25:07.469975 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:07.469981 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:07.470203 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:07.470339 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:25:07.470928 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:07.473123 (+ 2195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:07.474666 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:07.476384 (+ 1718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:07.476464 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:07.476589 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:07.476961 (+ 372us) server_negotiation.cc:299] Negotiation successful
0808 18:25:07.477051 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":123,"threads_started":1}
I20260808 18:25:07.477610 5812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:07.467118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:32961 (local address 127.0.0.1:41804)
0808 18:25:07.469590 (+ 2472us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:07.469605 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:07.469809 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:07.470567 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:07.470589 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:07.470614 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:25:07.470780 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:07.470788 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:07.473277 (+ 2489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:07.473280 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:07.474519 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:07.474530 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:07.476681 (+ 2151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:07.476684 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:07.476806 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:07.476828 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 18:25:07.477375 (+ 547us) client_negotiation.cc:240] Negotiation successful
0808 18:25:07.477448 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2408,"mutex_wait_us":1,"thread_start_us":165,"threads_started":1}
I20260808 18:25:07.480268 5808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:07.466922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.130:33909 (local address 127.0.0.1:35260)
0808 18:25:07.468989 (+ 2067us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:07.469040 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:07.469163 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:07.476126 (+ 6963us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:07.476129 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:07.476146 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:25:07.476358 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:07.476367 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:07.478031 (+ 1664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:07.478035 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:07.478913 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:07.478923 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:07.479831 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:07.479834 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:07.480000 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:07.480026 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 18:25:07.480074 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 18:25:07.480142 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1862,"thread_start_us":168,"threads_started":1}
I20260808 18:25:07.480135 5708 tablet_service.cc:1467] Processing CreateTablet for tablet 1754472f13bc4f9791dff6b57d57b9f0 (DEFAULT_TABLE table=test-table [id=19c47d97357c44f5be3651dc3d60916d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:25:07.480962 5810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:07.467827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35260 (local address 127.23.72.130:33909)
0808 18:25:07.475634 (+ 7807us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:07.475640 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:07.475663 (+ 23us) server_negotiation.cc:407] Connection header received
0808 18:25:07.475741 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:07.475746 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:07.475960 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:07.476519 (+ 559us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:25:07.476670 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:07.477876 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:07.479038 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:07.479674 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:07.480517 (+ 843us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:07.480650 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:07.480697 (+ 47us) server_negotiation.cc:299] Negotiation successful
0808 18:25:07.480777 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7700,"thread_start_us":130,"threads_started":1}
I20260808 18:25:07.481438 5708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1754472f13bc4f9791dff6b57d57b9f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:25:07.483840 5565 tablet_service.cc:1467] Processing CreateTablet for tablet 1754472f13bc4f9791dff6b57d57b9f0 (DEFAULT_TABLE table=test-table [id=19c47d97357c44f5be3651dc3d60916d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:25:07.484973 5565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1754472f13bc4f9791dff6b57d57b9f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:25:07.485515 5422 tablet_service.cc:1467] Processing CreateTablet for tablet 1754472f13bc4f9791dff6b57d57b9f0 (DEFAULT_TABLE table=test-table [id=19c47d97357c44f5be3651dc3d60916d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:25:07.487059 5422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1754472f13bc4f9791dff6b57d57b9f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:25:07.496033 5814 tablet_bootstrap.cc:492] T 1754472f13bc4f9791dff6b57d57b9f0 P 66ffdbc34d7e4c259b0d8c7938bdded9: Bootstrap starting.
I20260808 18:25:07.498994 5815 tablet_bootstrap.cc:492] T 1754472f13bc4f9791dff6b57d57b9f0 P 0c96af460dfa44a5a28964062a7ba43f: Bootstrap starting.
I20260808 18:25:07.499068 5814 tablet_bootstrap.cc:654] T 1754472f13bc4f9791dff6b57d57b9f0 P 66ffdbc34d7e4c259b0d8c7938bdded9: Neither blocks nor log segments found. Creating new log.
I20260808 18:25:07.500523 5814 log.cc:824] T 1754472f13bc4f9791dff6b57d57b9f0 P 66ffdbc34d7e4c259b0d8c7938bdded9: Log is configured to *not* fsync() on all Append() calls
I20260808 18:25:07.502810 5817 tablet_bootstrap.cc:492] T 1754472f13bc4f9791dff6b57d57b9f0 P 50cfb2e821534ee5b59c14854f08d212: Bootstrap starting.
I20260808 18:25:07.503000 5815 tablet_bootstrap.cc:654] T 1754472f13bc4f9791dff6b57d57b9f0 P 0c96af460dfa44a5a28964062a7ba43f: Neither blocks nor log segments found. Creating new log.
I20260808 18:25:07.503571 5814 tablet_bootstrap.cc:492] T 1754472f13bc4f9791dff6b57d57b9f0 P 66ffdbc34d7e4c259b0d8c7938bdded9: No bootstrap required, opened a new log
I20260808 18:25:07.503764 5814 ts_tablet_manager.cc:1397] T 1754472f13bc4f9791dff6b57d57b9f0 P 66ffdbc34d7e4c259b0d8c7938bdded9: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.007s
I20260808 18:25:07.504315 5815 log.cc:824] T 1754472f13bc4f9791dff6b57d57b9f0 P 0c96af460dfa44a5a28964062a7ba43f: Log is configured to *not* fsync() on all Append() calls
I20260808 18:25:07.506603 5817 tablet_bootstrap.cc:654] T 1754472f13bc4f9791dff6b57d57b9f0 P 50cfb2e821534ee5b59c14854f08d212: Neither blocks nor log segments found. Creating new log.
I20260808 18:25:07.508347 5817 log.cc:824] T 1754472f13bc4f9791dff6b57d57b9f0 P 50cfb2e821534ee5b59c14854f08d212: Log is configured to *not* fsync() on all Append() calls
I20260808 18:25:07.510895 5814 raft_consensus.cc:348] T 1754472f13bc4f9791dff6b57d57b9f0 P 66ffdbc34d7e4c259b0d8c7938bdded9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ffdbc34d7e4c259b0d8c7938bdded9" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 33909 } } peers { permanent_uuid: "0c96af460dfa44a5a28964062a7ba43f" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 32961 } } peers { permanent_uuid: "50cfb2e821534ee5b59c14854f08d212" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 35105 } }
I20260808 18:25:07.511313 5814 raft_consensus.cc:374] T 1754472f13bc4f9791dff6b57d57b9f0 P 66ffdbc34d7e4c259b0d8c7938bdded9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:25:07.511495 5814 raft_consensus.cc:729] T 1754472f13bc4f9791dff6b57d57b9f0 P 66ffdbc34d7e4c259b0d8c7938bdded9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66ffdbc34d7e4c259b0d8c7938bdded9, State: Initialized, Role: FOLLOWER
I20260808 18:25:07.511929 5815 tablet_bootstrap.cc:492] T 1754472f13bc4f9791dff6b57d57b9f0 P 0c96af460dfa44a5a28964062a7ba43f: No bootstrap required, opened a new log
I20260808 18:25:07.512140 5815 ts_tablet_manager.cc:1397] T 1754472f13bc4f9791dff6b57d57b9f0 P 0c96af460dfa44a5a28964062a7ba43f: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.004s
I20260808 18:25:07.512285 5814 consensus_queue.cc:260] T 1754472f13bc4f9791dff6b57d57b9f0 P 66ffdbc34d7e4c259b0d8c7938bdded9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ffdbc34d7e4c259b0d8c7938bdded9" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 33909 } } peers { permanent_uuid: "0c96af460dfa44a5a28964062a7ba43f" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 32961 } } peers { permanent_uuid: "50cfb2e821534ee5b59c14854f08d212" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 35105 } }
I20260808 18:25:07.513751 5814 ts_tablet_manager.cc:1428] T 1754472f13bc4f9791dff6b57d57b9f0 P 66ffdbc34d7e4c259b0d8c7938bdded9: Time spent starting tablet: real 0.010s user 0.005s sys 0.004s
I20260808 18:25:07.514225 5636 heartbeater.cc:503] Master 127.23.72.190:41867 was elected leader, sending a full tablet report...
I20260808 18:25:07.517678 5815 raft_consensus.cc:348] T 1754472f13bc4f9791dff6b57d57b9f0 P 0c96af460dfa44a5a28964062a7ba43f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ffdbc34d7e4c259b0d8c7938bdded9" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 33909 } } peers { permanent_uuid: "0c96af460dfa44a5a28964062a7ba43f" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 32961 } } peers { permanent_uuid: "50cfb2e821534ee5b59c14854f08d212" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 35105 } }
I20260808 18:25:07.517993 5815 raft_consensus.cc:374] T 1754472f13bc4f9791dff6b57d57b9f0 P 0c96af460dfa44a5a28964062a7ba43f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:25:07.518091 5815 raft_consensus.cc:729] T 1754472f13bc4f9791dff6b57d57b9f0 P 0c96af460dfa44a5a28964062a7ba43f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c96af460dfa44a5a28964062a7ba43f, State: Initialized, Role: FOLLOWER
I20260808 18:25:07.518600 5815 consensus_queue.cc:260] T 1754472f13bc4f9791dff6b57d57b9f0 P 0c96af460dfa44a5a28964062a7ba43f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ffdbc34d7e4c259b0d8c7938bdded9" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 33909 } } peers { permanent_uuid: "0c96af460dfa44a5a28964062a7ba43f" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 32961 } } peers { permanent_uuid: "50cfb2e821534ee5b59c14854f08d212" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 35105 } }
I20260808 18:25:07.520124 5815 ts_tablet_manager.cc:1428] T 1754472f13bc4f9791dff6b57d57b9f0 P 0c96af460dfa44a5a28964062a7ba43f: Time spent starting tablet: real 0.008s user 0.005s sys 0.003s
I20260808 18:25:07.520509 5491 heartbeater.cc:503] Master 127.23.72.190:41867 was elected leader, sending a full tablet report...
I20260808 18:25:07.521533 5817 tablet_bootstrap.cc:492] T 1754472f13bc4f9791dff6b57d57b9f0 P 50cfb2e821534ee5b59c14854f08d212: No bootstrap required, opened a new log
I20260808 18:25:07.521740 5817 ts_tablet_manager.cc:1397] T 1754472f13bc4f9791dff6b57d57b9f0 P 50cfb2e821534ee5b59c14854f08d212: Time spent bootstrapping tablet: real 0.019s user 0.006s sys 0.000s
I20260808 18:25:07.528381 5817 raft_consensus.cc:348] T 1754472f13bc4f9791dff6b57d57b9f0 P 50cfb2e821534ee5b59c14854f08d212 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ffdbc34d7e4c259b0d8c7938bdded9" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 33909 } } peers { permanent_uuid: "0c96af460dfa44a5a28964062a7ba43f" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 32961 } } peers { permanent_uuid: "50cfb2e821534ee5b59c14854f08d212" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 35105 } }
I20260808 18:25:07.528700 5817 raft_consensus.cc:374] T 1754472f13bc4f9791dff6b57d57b9f0 P 50cfb2e821534ee5b59c14854f08d212 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:25:07.528796 5817 raft_consensus.cc:729] T 1754472f13bc4f9791dff6b57d57b9f0 P 50cfb2e821534ee5b59c14854f08d212 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50cfb2e821534ee5b59c14854f08d212, State: Initialized, Role: FOLLOWER
I20260808 18:25:07.529279 5817 consensus_queue.cc:260] T 1754472f13bc4f9791dff6b57d57b9f0 P 50cfb2e821534ee5b59c14854f08d212 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ffdbc34d7e4c259b0d8c7938bdded9" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 33909 } } peers { permanent_uuid: "0c96af460dfa44a5a28964062a7ba43f" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 32961 } } peers { permanent_uuid: "50cfb2e821534ee5b59c14854f08d212" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 35105 } }
I20260808 18:25:07.530260 5817 ts_tablet_manager.cc:1428] T 1754472f13bc4f9791dff6b57d57b9f0 P 50cfb2e821534ee5b59c14854f08d212: Time spent starting tablet: real 0.008s user 0.009s sys 0.000s
I20260808 18:25:07.530615 5778 heartbeater.cc:503] Master 127.23.72.190:41867 was elected leader, sending a full tablet report...
Aug 08 18:25:07 dist-test-slave-5k54 krb5kdc[4215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213506, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for HTTP/127.23.72.148@KRBTEST.COM
I20260808 18:25:07.614002 5824 raft_consensus.cc:482] T 1754472f13bc4f9791dff6b57d57b9f0 P 50cfb2e821534ee5b59c14854f08d212 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:25:07.614253 5824 raft_consensus.cc:504] T 1754472f13bc4f9791dff6b57d57b9f0 P 50cfb2e821534ee5b59c14854f08d212 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ffdbc34d7e4c259b0d8c7938bdded9" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 33909 } } peers { permanent_uuid: "0c96af460dfa44a5a28964062a7ba43f" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 32961 } } peers { permanent_uuid: "50cfb2e821534ee5b59c14854f08d212" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 35105 } }
I20260808 18:25:07.615880 5824 leader_election.cc:288] T 1754472f13bc4f9791dff6b57d57b9f0 P 50cfb2e821534ee5b59c14854f08d212 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66ffdbc34d7e4c259b0d8c7938bdded9 (127.23.72.130:33909), 0c96af460dfa44a5a28964062a7ba43f (127.23.72.129:32961)
W20260808 18:25:07.626447 5492 tablet.cc:2369] T 1754472f13bc4f9791dff6b57d57b9f0 P 0c96af460dfa44a5a28964062a7ba43f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:25:07.629752 5810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:07.616486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:52083 (local address 127.23.72.130:33909)
0808 18:25:07.616600 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:07.616604 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:07.622195 (+ 5591us) server_negotiation.cc:407] Connection header received
0808 18:25:07.622423 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:07.622427 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:07.622477 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:07.622550 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:25:07.624278 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:07.625073 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:07.628281 (+ 3208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:07.628908 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:07.629052 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:07.629109 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:07.629524 (+ 415us) server_negotiation.cc:299] Negotiation successful
0808 18:25:07.629616 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 18:25:07.630231 5827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:07.616079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.130:33909 (local address 127.23.72.131:52083)
0808 18:25:07.622025 (+ 5946us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:07.622046 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:07.622297 (+ 251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:07.622789 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:07.622793 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:07.622818 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:25:07.623132 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:07.623141 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:07.625228 (+ 2087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:07.625232 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:07.628083 (+ 2851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:07.628099 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:07.629180 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:07.629183 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:07.629344 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:07.629371 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 18:25:07.629948 (+ 577us) client_negotiation.cc:240] Negotiation successful
0808 18:25:07.630032 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4960,"thread_start_us":138,"threads_started":1}
I20260808 18:25:07.631073 5586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1754472f13bc4f9791dff6b57d57b9f0" candidate_uuid: "50cfb2e821534ee5b59c14854f08d212" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ffdbc34d7e4c259b0d8c7938bdded9" is_pre_election: true
I20260808 18:25:07.631457 5586 raft_consensus.cc:2454] T 1754472f13bc4f9791dff6b57d57b9f0 P 66ffdbc34d7e4c259b0d8c7938bdded9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50cfb2e821534ee5b59c14854f08d212 in term 0.
I20260808 18:25:07.632128 5660 leader_election.cc:302] T 1754472f13bc4f9791dff6b57d57b9f0 P 50cfb2e821534ee5b59c14854f08d212 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50cfb2e821534ee5b59c14854f08d212, 66ffdbc34d7e4c259b0d8c7938bdded9; no voters:
I20260808 18:25:07.632455 5824 raft_consensus.cc:2781] T 1754472f13bc4f9791dff6b57d57b9f0 P 50cfb2e821534ee5b59c14854f08d212 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:25:07.632578 5824 raft_consensus.cc:482] T 1754472f13bc4f9791dff6b57d57b9f0 P 50cfb2e821534ee5b59c14854f08d212 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:25:07.632622 5824 raft_consensus.cc:3037] T 1754472f13bc4f9791dff6b57d57b9f0 P 50cfb2e821534ee5b59c14854f08d212 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:25:07.634400 5824 raft_consensus.cc:504] T 1754472f13bc4f9791dff6b57d57b9f0 P 50cfb2e821534ee5b59c14854f08d212 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ffdbc34d7e4c259b0d8c7938bdded9" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 33909 } } peers { permanent_uuid: "0c96af460dfa44a5a28964062a7ba43f" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 32961 } } peers { permanent_uuid: "50cfb2e821534ee5b59c14854f08d212" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 35105 } }
I20260808 18:25:07.634809 5824 leader_election.cc:288] T 1754472f13bc4f9791dff6b57d57b9f0 P 50cfb2e821534ee5b59c14854f08d212 [CANDIDATE]: Term 1 election: Requested vote from peers 66ffdbc34d7e4c259b0d8c7938bdded9 (127.23.72.130:33909), 0c96af460dfa44a5a28964062a7ba43f (127.23.72.129:32961)
I20260808 18:25:07.635453 5586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1754472f13bc4f9791dff6b57d57b9f0" candidate_uuid: "50cfb2e821534ee5b59c14854f08d212" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ffdbc34d7e4c259b0d8c7938bdded9"
I20260808 18:25:07.635592 5586 raft_consensus.cc:3037] T 1754472f13bc4f9791dff6b57d57b9f0 P 66ffdbc34d7e4c259b0d8c7938bdded9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:25:07.637254 5586 raft_consensus.cc:2454] T 1754472f13bc4f9791dff6b57d57b9f0 P 66ffdbc34d7e4c259b0d8c7938bdded9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50cfb2e821534ee5b59c14854f08d212 in term 1.
W20260808 18:25:07.637573 5637 tablet.cc:2369] T 1754472f13bc4f9791dff6b57d57b9f0 P 66ffdbc34d7e4c259b0d8c7938bdded9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:25:07.637688 5660 leader_election.cc:302] T 1754472f13bc4f9791dff6b57d57b9f0 P 50cfb2e821534ee5b59c14854f08d212 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50cfb2e821534ee5b59c14854f08d212, 66ffdbc34d7e4c259b0d8c7938bdded9; no voters:
I20260808 18:25:07.638813 5824 raft_consensus.cc:2781] T 1754472f13bc4f9791dff6b57d57b9f0 P 50cfb2e821534ee5b59c14854f08d212 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:25:07.639043 5824 raft_consensus.cc:686] T 1754472f13bc4f9791dff6b57d57b9f0 P 50cfb2e821534ee5b59c14854f08d212 [term 1 LEADER]: Becoming Leader. State: Replica: 50cfb2e821534ee5b59c14854f08d212, State: Running, Role: LEADER
I20260808 18:25:07.639391 5824 consensus_queue.cc:237] T 1754472f13bc4f9791dff6b57d57b9f0 P 50cfb2e821534ee5b59c14854f08d212 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ffdbc34d7e4c259b0d8c7938bdded9" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 33909 } } peers { permanent_uuid: "0c96af460dfa44a5a28964062a7ba43f" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 32961 } } peers { permanent_uuid: "50cfb2e821534ee5b59c14854f08d212" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 35105 } }
I20260808 18:25:07.643198 5278 catalog_manager.cc:5495] T 1754472f13bc4f9791dff6b57d57b9f0 P 50cfb2e821534ee5b59c14854f08d212 reported cstate change: term changed from 0 to 1, leader changed from <none> to 50cfb2e821534ee5b59c14854f08d212 (127.23.72.131). New cstate: current_term: 1 leader_uuid: "50cfb2e821534ee5b59c14854f08d212" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ffdbc34d7e4c259b0d8c7938bdded9" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c96af460dfa44a5a28964062a7ba43f" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 32961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50cfb2e821534ee5b59c14854f08d212" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 35105 } health_report { overall_health: HEALTHY } } }
W20260808 18:25:07.644613 5779 tablet.cc:2369] T 1754472f13bc4f9791dff6b57d57b9f0 P 50cfb2e821534ee5b59c14854f08d212: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:25:07.646756 5822 raft_consensus.cc:482] T 1754472f13bc4f9791dff6b57d57b9f0 P 0c96af460dfa44a5a28964062a7ba43f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:25:07.646952 5822 raft_consensus.cc:504] T 1754472f13bc4f9791dff6b57d57b9f0 P 0c96af460dfa44a5a28964062a7ba43f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ffdbc34d7e4c259b0d8c7938bdded9" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 33909 } } peers { permanent_uuid: "0c96af460dfa44a5a28964062a7ba43f" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 32961 } } peers { permanent_uuid: "50cfb2e821534ee5b59c14854f08d212" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 35105 } }
I20260808 18:25:07.648347 5822 leader_election.cc:288] T 1754472f13bc4f9791dff6b57d57b9f0 P 0c96af460dfa44a5a28964062a7ba43f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66ffdbc34d7e4c259b0d8c7938bdded9 (127.23.72.130:33909), 50cfb2e821534ee5b59c14854f08d212 (127.23.72.131:35105)
I20260808 18:25:07.651589 5811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:07.616708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:59847 (local address 127.23.72.129:32961)
0808 18:25:07.616864 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:07.616869 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:07.630519 (+ 13650us) server_negotiation.cc:407] Connection header received
0808 18:25:07.630720 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:07.630723 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:07.630772 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:07.630819 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:25:07.631313 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:07.632183 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:07.637943 (+ 5760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:07.638574 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:07.638696 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:07.638748 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:07.651343 (+ 12595us) server_negotiation.cc:299] Negotiation successful
0808 18:25:07.651461 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 18:25:07.656005 5826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:07.616280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:32961 (local address 127.23.72.131:59847)
0808 18:25:07.630293 (+ 14013us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:07.630409 (+ 116us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:07.630601 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:07.630945 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:07.630948 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:07.630964 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:25:07.631173 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:07.631180 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:07.635851 (+ 4671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:07.635854 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:07.637212 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:07.637224 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:07.650696 (+ 13472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:07.650703 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:07.650882 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:07.650912 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 18:25:07.655651 (+ 4739us) client_negotiation.cc:240] Negotiation successful
0808 18:25:07.655784 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13937,"thread_start_us":146,"threads_started":1}
I20260808 18:25:07.657011 5443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1754472f13bc4f9791dff6b57d57b9f0" candidate_uuid: "50cfb2e821534ee5b59c14854f08d212" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c96af460dfa44a5a28964062a7ba43f" is_pre_election: true
I20260808 18:25:07.657275 5443 raft_consensus.cc:2454] T 1754472f13bc4f9791dff6b57d57b9f0 P 0c96af460dfa44a5a28964062a7ba43f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50cfb2e821534ee5b59c14854f08d212 in term 0.
I20260808 18:25:07.657671 5442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1754472f13bc4f9791dff6b57d57b9f0" candidate_uuid: "50cfb2e821534ee5b59c14854f08d212" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c96af460dfa44a5a28964062a7ba43f"
I20260808 18:25:07.657775 5442 raft_consensus.cc:3037] T 1754472f13bc4f9791dff6b57d57b9f0 P 0c96af460dfa44a5a28964062a7ba43f [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:25:07.659686 5442 raft_consensus.cc:2454] T 1754472f13bc4f9791dff6b57d57b9f0 P 0c96af460dfa44a5a28964062a7ba43f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50cfb2e821534ee5b59c14854f08d212 in term 1.
I20260808 18:25:07.672214 5810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:07.649948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:34761 (local address 127.23.72.130:33909)
0808 18:25:07.650236 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:07.650241 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:07.660981 (+ 10740us) server_negotiation.cc:407] Connection header received
0808 18:25:07.661177 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:07.661183 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:07.661271 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:07.661348 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:25:07.664888 (+ 3540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:07.665706 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:07.670690 (+ 4984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:07.671406 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:07.671524 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:07.671578 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:07.671992 (+ 414us) server_negotiation.cc:299] Negotiation successful
0808 18:25:07.672071 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203}
I20260808 18:25:07.672930 5831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:07.648862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.130:33909 (local address 127.23.72.129:34761)
0808 18:25:07.660884 (+ 12022us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:07.660901 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:07.661057 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:07.664265 (+ 3208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:07.664269 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:07.664321 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:25:07.664663 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:07.664680 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:07.666022 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:07.666025 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:07.667445 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:07.667458 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:07.671672 (+ 4214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:07.671676 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:07.671846 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:07.671870 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 18:25:07.672611 (+ 741us) client_negotiation.cc:240] Negotiation successful
0808 18:25:07.672695 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11942,"thread_start_us":167,"threads_started":1}
I20260808 18:25:07.673560 5586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1754472f13bc4f9791dff6b57d57b9f0" candidate_uuid: "0c96af460dfa44a5a28964062a7ba43f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ffdbc34d7e4c259b0d8c7938bdded9" is_pre_election: true
I20260808 18:25:07.673800 5586 raft_consensus.cc:2379] T 1754472f13bc4f9791dff6b57d57b9f0 P 66ffdbc34d7e4c259b0d8c7938bdded9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0c96af460dfa44a5a28964062a7ba43f in current term 1: Already voted for candidate 50cfb2e821534ee5b59c14854f08d212 in this term.
I20260808 18:25:07.676381 5813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:07.650329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:43021 (local address 127.23.72.131:35105)
0808 18:25:07.650429 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:07.650433 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:07.665479 (+ 15046us) server_negotiation.cc:407] Connection header received
0808 18:25:07.667609 (+ 2130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:07.667613 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:07.667661 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:07.667706 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:25:07.668221 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:07.668935 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:07.674660 (+ 5725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:07.675279 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:07.675574 (+ 295us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:07.675624 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:07.676188 (+ 564us) server_negotiation.cc:299] Negotiation successful
0808 18:25:07.676264 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":41}
I20260808 18:25:07.676729 5833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:07.663764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52088 (local address 127.23.72.131:35105)
0808 18:25:07.670393 (+ 6629us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:07.670416 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:07.670430 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:25:07.670503 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:07.670506 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:07.670574 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:07.671069 (+ 495us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:25:07.671156 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:07.672125 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:07.673474 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:07.673827 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:07.673967 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:07.674564 (+ 597us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:25:07.675517 (+ 953us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:25:07.676358 (+ 841us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:07.676397 (+ 39us) server_negotiation.cc:299] Negotiation successful
0808 18:25:07.676457 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6552,"thread_start_us":270,"threads_started":1}
I20260808 18:25:07.676972 5830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:07.649312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.131:35105 (local address 127.23.72.129:43021)
0808 18:25:07.665338 (+ 16026us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:07.665362 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:07.665831 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:07.667829 (+ 1998us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:07.667832 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:07.667848 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:25:07.668094 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:07.668101 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:07.669076 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:07.669079 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:07.670522 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:07.670531 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:07.675733 (+ 5202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:07.675736 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:07.676030 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:07.676056 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 18:25:07.676706 (+ 650us) client_negotiation.cc:240] Negotiation successful
0808 18:25:07.676782 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15893,"thread_start_us":127,"threads_started":1}
I20260808 18:25:07.678172 5728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1754472f13bc4f9791dff6b57d57b9f0" candidate_uuid: "0c96af460dfa44a5a28964062a7ba43f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50cfb2e821534ee5b59c14854f08d212" is_pre_election: true
I20260808 18:25:07.678932 5376 leader_election.cc:302] T 1754472f13bc4f9791dff6b57d57b9f0 P 0c96af460dfa44a5a28964062a7ba43f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0c96af460dfa44a5a28964062a7ba43f; no voters: 50cfb2e821534ee5b59c14854f08d212, 66ffdbc34d7e4c259b0d8c7938bdded9
I20260808 18:25:07.679488 5822 raft_consensus.cc:2726] T 1754472f13bc4f9791dff6b57d57b9f0 P 0c96af460dfa44a5a28964062a7ba43f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 18:25:07.684109 5442 raft_consensus.cc:1261] T 1754472f13bc4f9791dff6b57d57b9f0 P 0c96af460dfa44a5a28964062a7ba43f [term 1 FOLLOWER]: Refusing update from remote peer 50cfb2e821534ee5b59c14854f08d212: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:25:07.684118 5586 raft_consensus.cc:1261] T 1754472f13bc4f9791dff6b57d57b9f0 P 66ffdbc34d7e4c259b0d8c7938bdded9 [term 1 FOLLOWER]: Refusing update from remote peer 50cfb2e821534ee5b59c14854f08d212: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:25:07.684787 5824 consensus_queue.cc:1035] T 1754472f13bc4f9791dff6b57d57b9f0 P 50cfb2e821534ee5b59c14854f08d212 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c96af460dfa44a5a28964062a7ba43f" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 32961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:25:07.685087 5828 consensus_queue.cc:1035] T 1754472f13bc4f9791dff6b57d57b9f0 P 50cfb2e821534ee5b59c14854f08d212 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66ffdbc34d7e4c259b0d8c7938bdded9" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:25:07.702255 5835 mvcc.cc:205] Tried to move back new op lower bound from 7316330527467008000 to 7316330527294562304. Current Snapshot: MvccSnapshot[applied={T|T < 7316330527467008000}]
I20260808 18:25:07.703219 5838 mvcc.cc:205] Tried to move back new op lower bound from 7316330527467008000 to 7316330527294562304. Current Snapshot: MvccSnapshot[applied={T|T < 7316330527467008000}]
I20260808 18:25:07.707573 5837 mvcc.cc:205] Tried to move back new op lower bound from 7316330527467008000 to 7316330527294562304. Current Snapshot: MvccSnapshot[applied={T|T < 7316330527467008000}]
I20260808 18:25:07.714700 5279 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:25:07.714920 5279 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:25:07.722708 5279 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 874ed22651ea4961bc0120df29e12100: Sending DeleteTablet for 3 replicas of tablet 1754472f13bc4f9791dff6b57d57b9f0
I20260808 18:25:07.724133 5708 tablet_service.cc:1514] Processing DeleteTablet for tablet 1754472f13bc4f9791dff6b57d57b9f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:25:07 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:52078
I20260808 18:25:07.724748 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 5320
I20260808 18:25:07.724812 5422 tablet_service.cc:1514] Processing DeleteTablet for tablet 1754472f13bc4f9791dff6b57d57b9f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:25:07 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:41804
I20260808 18:25:07.725749 5565 tablet_service.cc:1514] Processing DeleteTablet for tablet 1754472f13bc4f9791dff6b57d57b9f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:25:07 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:35260
I20260808 18:25:07.730300 5845 tablet_replica.cc:330] T 1754472f13bc4f9791dff6b57d57b9f0 P 50cfb2e821534ee5b59c14854f08d212: stopping tablet replica
I20260808 18:25:07.731413 5845 raft_consensus.cc:2229] T 1754472f13bc4f9791dff6b57d57b9f0 P 50cfb2e821534ee5b59c14854f08d212 [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:25:07.731949 5845 raft_consensus.cc:2258] T 1754472f13bc4f9791dff6b57d57b9f0 P 50cfb2e821534ee5b59c14854f08d212 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:25:07.733695 5845 ts_tablet_manager.cc:1905] T 1754472f13bc4f9791dff6b57d57b9f0 P 50cfb2e821534ee5b59c14854f08d212: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:25:07.737941 5845 ts_tablet_manager.cc:1918] T 1754472f13bc4f9791dff6b57d57b9f0 P 50cfb2e821534ee5b59c14854f08d212: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:25:07.738130 5845 log.cc:1196] T 1754472f13bc4f9791dff6b57d57b9f0 P 50cfb2e821534ee5b59c14854f08d212: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213345431840-23842-0/minicluster-data/ts-2/wal/wals/1754472f13bc4f9791dff6b57d57b9f0
I20260808 18:25:07.738591 5845 ts_tablet_manager.cc:1939] T 1754472f13bc4f9791dff6b57d57b9f0 P 50cfb2e821534ee5b59c14854f08d212: Deleting consensus metadata
W20260808 18:25:07.739234 5265 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.72.129:32961 (error 108)
I20260808 18:25:07.740195 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 5493
I20260808 18:25:07.744722 5263 catalog_manager.cc:4841] TS 50cfb2e821534ee5b59c14854f08d212 (127.23.72.131:35105): tablet 1754472f13bc4f9791dff6b57d57b9f0 (table test-table [id=19c47d97357c44f5be3651dc3d60916d]) successfully deleted
I20260808 18:25:07.745913 5808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:07.745628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:32961 (local address 127.0.0.1:41816)
0808 18:25:07.745761 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:07.745839 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.129:32961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 18:25:07.746182 5265 catalog_manager.cc:4584] TS 0c96af460dfa44a5a28964062a7ba43f (127.23.72.129:32961): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1754472f13bc4f9791dff6b57d57b9f0: Network error: Client connection negotiation failed: client connection to 127.23.72.129:32961: connect: Connection refused (error 111)
I20260808 18:25:07.755185 5808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:07.754737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.130:33909 (local address 127.0.0.1:35268)
0808 18:25:07.754968 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:07.755069 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.130:33909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143}
I20260808 18:25:07.755453 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 5638
I20260808 18:25:07.772085 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 5243
I20260808 18:25:07.782698 23842 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 18:25:08.127548 23842 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 18:25:08.127671 23842 mini_ranger.cc:66] Stopping Ranger...
I20260808 18:25:08.472548 23842 mini_ranger.cc:68] Stopped Ranger
2026-08-08 18:25:08.472 UTC [4248] LOG: received smart shutdown request
2026-08-08 18:25:08.476 UTC [4248] LOG: background worker "logical replication launcher" (PID 4258) exited with exit code 1
2026-08-08 18:25:08.477 UTC [4253] LOG: shutting down
2026-08-08 18:25:08.655 UTC [4248] LOG: database system is shut down
2026-08-08T18:25:08Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (49657 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:25:08 dist-test-slave-5k54 krb5kdc[5856](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:25:08 dist-test-slave-5k54 krb5kdc[5856](info): set up 2 sockets
Aug 08 18:25:08 dist-test-slave-5k54 krb5kdc[5856](info): commencing operation
krb5kdc: starting...
W20260808 18:25:09.868733 23842 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Aug 08 18:25:09 dist-test-slave-5k54 krb5kdc[5856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:25:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:25:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.72.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
I20260808 18:25:09.996246 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:33023
--webserver_interface=127.23.72.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:35725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:33023,127.23.72.189:36153,127.23.72.188:36963
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:25:10.107698 5869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:25:10.108078 5869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:25:10.108211 5869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:25:10.112346 5869 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:25:10.112416 5869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:25:10.112435 5869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:25:10.112452 5869 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:25:10.112468 5869 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:25:10.117923 5869 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:35725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:33023,127.23.72.189:36153,127.23.72.188:36963
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:33023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.72.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5869
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:25:10.119020 5869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:25:10.120106 5869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:25:10.128680 5874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:10.128692 5877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:25:10.128732 5869 server_base.cc:1034] running on GCE node
W20260808 18:25:10.128692 5875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:25:10.129464 5869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:25:10.130556 5869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:25:10.131742 5869 hybrid_clock.cc:648] HybridClock initialized: now 1786213510131727 us; error 25 us; skew 500 ppm
Aug 08 18:25:10 dist-test-slave-5k54 krb5kdc[5856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213510, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:25:10.134663 5869 init.cc:377] Logged in from keytab as kudu/127.23.72.190@KRBTEST.COM (short username kudu)
I20260808 18:25:10.135762 5869 webserver.cc:460] Webserver started at http://127.23.72.190:38515/ using document root <none> and password file <none>
I20260808 18:25:10.136376 5869 fs_manager.cc:359] Metadata directory not provided
I20260808 18:25:10.136482 5869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:25:10.136735 5869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:25:10.138939 5869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "bf8eea685e974e0b820d809c987df684"
format_stamp: "Formatted at 2026-08-08 18:25:10 on dist-test-slave-5k54"
server_key: "a0bc231a7c52fa8d1a68779ed75d6e51"
server_key_iv: "bf528f505cd2200ba5c2efb19e4a90d7"
server_key_version: "encryptionkey@0"
I20260808 18:25:10.139636 5869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "bf8eea685e974e0b820d809c987df684"
format_stamp: "Formatted at 2026-08-08 18:25:10 on dist-test-slave-5k54"
server_key: "a0bc231a7c52fa8d1a68779ed75d6e51"
server_key_iv: "bf528f505cd2200ba5c2efb19e4a90d7"
server_key_version: "encryptionkey@0"
I20260808 18:25:10.143697 5869 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 18:25:10.146266 5884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:10.147584 5869 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:25:10.147717 5869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "bf8eea685e974e0b820d809c987df684"
format_stamp: "Formatted at 2026-08-08 18:25:10 on dist-test-slave-5k54"
server_key: "a0bc231a7c52fa8d1a68779ed75d6e51"
server_key_iv: "bf528f505cd2200ba5c2efb19e4a90d7"
server_key_version: "encryptionkey@0"
I20260808 18:25:10.147866 5869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:25:10.169663 5869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:25:10.172643 5869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:25:10.172852 5869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:25:10.180250 5869 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.190:33023
I20260808 18:25:10.180321 5940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.190:33023 every 8 connection(s)
I20260808 18:25:10.181265 5869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:25:10.183009 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 5869
I20260808 18:25:10.183140 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:25:10.183481 23842 external_mini_cluster.cc:1428] Setting key 8a9609305678d0a730425db4fd77447b
I20260808 18:25:10.184247 5941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:25:10.188102 5941 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.72.190" port: 33023 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.23.72.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.189@KRBTEST.COM" created.
Aug 08 18:25:10 dist-test-slave-5k54 krb5kdc[5856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213510, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:25:10.204023 5944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:10.190431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57246 (local address 127.23.72.190:33023)
0808 18:25:10.190860 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:10.190880 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:10.191144 (+ 264us) server_negotiation.cc:407] Connection header received
0808 18:25:10.192216 (+ 1072us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:10.192220 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:10.192450 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:10.192650 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:25:10.193527 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:10.194467 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:10.195671 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:10.196045 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:10.196214 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:10.199107 (+ 2893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:25:10.199113 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:25:10.199122 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:25:10.199144 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:25:10.200893 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:10.201522 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:10.201527 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:10.201528 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:10.201639 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:10.202030 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:10.202033 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:10.202034 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:10.202397 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:25:10.202503 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:10.202971 (+ 468us) server_negotiation.cc:299] Negotiation successful
0808 18:25:10.203204 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":277,"thread_start_us":146,"threads_started":1}
I20260808 18:25:10.204028 5945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:10.190431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:33023 (local address 127.0.0.1:57246)
0808 18:25:10.191001 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:10.191037 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:10.191911 (+ 874us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:10.192648 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:10.192653 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:10.193073 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:25:10.193382 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:10.193397 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:10.194599 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:10.194602 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:10.195558 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:10.195568 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:10.196202 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:10.196206 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:10.196327 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:10.197016 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:25:10.197042 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:25:10.198925 (+ 1883us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:25:10.201015 (+ 2090us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:10.201028 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:10.201042 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:10.201387 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:10.201739 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:10.201744 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:10.201747 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:10.201909 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:10.202524 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:25:10.202534 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:25:10.202703 (+ 169us) client_negotiation.cc:769] Sending connection context
0808 18:25:10.202905 (+ 202us) client_negotiation.cc:240] Negotiation successful
0808 18:25:10.203314 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"mutex_wait_us":82,"spinlock_wait_cycles":15232,"thread_start_us":163,"threads_started":1}
I20260808 18:25:10.206467 5941 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.72.189" port: 36153 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:25:10.207206 5945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:10.206935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.189:36153 (local address 127.0.0.1:35864)
0808 18:25:10.207048 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:10.207135 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.189:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 18:25:10.207382 5886 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.72.189:36153: connect: Connection refused (error 111)
I20260808 18:25:10.208294 5945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:10.208091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.189:36153 (local address 127.0.0.1:35870)
0808 18:25:10.208190 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:10.208234 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.189:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 18:25:10.208448 5941 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.72.189:36153: Network error: Client connection negotiation failed: client connection to 127.23.72.189:36153: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.189.keytab.
Entry for principal kudu/127.23.72.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.189.keytab.
I20260808 18:25:10.212651 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.189:36153
--webserver_interface=127.23.72.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.189.keytab
--principal=kudu/127.23.72.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:35725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:33023,127.23.72.189:36153,127.23.72.188:36963
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:25:10.257704 5941 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.72.189" port: 36153 } attempt: 1
I20260808 18:25:10.258625 5945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:10.258315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.189:36153 (local address 127.0.0.1:35874)
0808 18:25:10.258479 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:10.258546 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.189:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260808 18:25:10.259351 5945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:10.259158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.189:36153 (local address 127.0.0.1:35890)
0808 18:25:10.259254 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:10.259303 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.189:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 18:25:10.259503 5941 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.72.189:36153: Network error: Client connection negotiation failed: client connection to 127.23.72.189:36153: connect: Connection refused (error 111)
W20260808 18:25:10.323849 5948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:25:10.324113 5948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:25:10.324152 5948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 18:25:10.327675 5941 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.72.189" port: 36153 } attempt: 2
W20260808 18:25:10.328351 5948 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:25:10.328425 5948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:25:10.328447 5948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:25:10.328464 5948 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:25:10.328514 5948 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:25:10.328675 5945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:10.328245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.189:36153 (local address 127.0.0.1:35898)
0808 18:25:10.328496 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:10.328598 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.189:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260808 18:25:10.329443 5945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:10.329216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.189:36153 (local address 127.0.0.1:35904)
0808 18:25:10.329325 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:10.329389 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.189:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 18:25:10.329619 5941 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.72.189:36153: Network error: Client connection negotiation failed: client connection to 127.23.72.189:36153: connect: Connection refused (error 111)
I20260808 18:25:10.333946 5948 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:35725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:33023,127.23.72.189:36153,127.23.72.188:36963
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.189.keytab
--principal=kudu/127.23.72.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.189:36153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.23.72.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5948
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:25:10.334970 5948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:25:10.336025 5948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:25:10.344558 5954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:10.344558 5953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:10.344583 5957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:25:10.344635 5948 server_base.cc:1034] running on GCE node
I20260808 18:25:10.345321 5948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:25:10.346432 5948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:25:10.347609 5948 hybrid_clock.cc:648] HybridClock initialized: now 1786213510347590 us; error 30 us; skew 500 ppm
Aug 08 18:25:10 dist-test-slave-5k54 krb5kdc[5856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213510, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:25:10.351331 5948 init.cc:377] Logged in from keytab as kudu/127.23.72.189@KRBTEST.COM (short username kudu)
I20260808 18:25:10.352382 5948 webserver.cc:460] Webserver started at http://127.23.72.189:40665/ using document root <none> and password file <none>
I20260808 18:25:10.352984 5948 fs_manager.cc:359] Metadata directory not provided
I20260808 18:25:10.353049 5948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:25:10.353273 5948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:25:10.355288 5948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-1/data/instance:
uuid: "4d59fd403b144375a6c82b46e0bfcc33"
format_stamp: "Formatted at 2026-08-08 18:25:10 on dist-test-slave-5k54"
server_key: "ab96c316c2cdeb1ab89560f730cab537"
server_key_iv: "92f42de61ec82aa13ab6879090b489f0"
server_key_version: "encryptionkey@0"
I20260808 18:25:10.355924 5948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-1/wal/instance:
uuid: "4d59fd403b144375a6c82b46e0bfcc33"
format_stamp: "Formatted at 2026-08-08 18:25:10 on dist-test-slave-5k54"
server_key: "ab96c316c2cdeb1ab89560f730cab537"
server_key_iv: "92f42de61ec82aa13ab6879090b489f0"
server_key_version: "encryptionkey@0"
I20260808 18:25:10.359568 5948 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:25:10.361905 5964 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:10.363061 5948 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:25:10.363199 5948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-1/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-1/wal
uuid: "4d59fd403b144375a6c82b46e0bfcc33"
format_stamp: "Formatted at 2026-08-08 18:25:10 on dist-test-slave-5k54"
server_key: "ab96c316c2cdeb1ab89560f730cab537"
server_key_iv: "92f42de61ec82aa13ab6879090b489f0"
server_key_version: "encryptionkey@0"
I20260808 18:25:10.363309 5948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:25:10.375357 5948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:25:10.378564 5948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:25:10.378759 5948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:25:10.385752 5948 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.189:36153
I20260808 18:25:10.385805 6017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.189:36153 every 8 connection(s)
I20260808 18:25:10.386762 5948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-1/data/info.pb
I20260808 18:25:10.389245 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 5948
I20260808 18:25:10.389392 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-1/wal/instance
I20260808 18:25:10.389724 23842 external_mini_cluster.cc:1428] Setting key 81bce93ce8e7c13092bf4add1ae09f1d
I20260808 18:25:10.389770 6018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:25:10.393946 6018 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.72.190" port: 33023 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.23.72.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.188@KRBTEST.COM" created.
Aug 08 18:25:10 dist-test-slave-5k54 krb5kdc[5856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213510, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.189@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:25:10.407990 5944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:10.395870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57250 (local address 127.23.72.190:33023)
0808 18:25:10.396050 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:10.396056 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:10.396815 (+ 759us) server_negotiation.cc:407] Connection header received
0808 18:25:10.397747 (+ 932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:10.397752 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:10.397828 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:10.397950 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:25:10.399219 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:10.399820 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:10.400943 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:10.401195 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:10.401307 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:10.403946 (+ 2639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:25:10.403953 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:25:10.403958 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:25:10.403992 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:25:10.405761 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:10.406273 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:10.406277 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:10.406282 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:10.406394 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:10.406721 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:10.406728 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:10.406730 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:10.406999 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:25:10.407131 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:10.407639 (+ 508us) server_negotiation.cc:299] Negotiation successful
0808 18:25:10.407773 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 18:25:10.408735 6021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:10.396199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:33023 (local address 127.0.0.1:57250)
0808 18:25:10.396651 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:10.396699 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:10.397530 (+ 831us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:10.398135 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:10.398142 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:10.398540 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:25:10.399012 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:10.399025 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:10.399962 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:10.399965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:10.400800 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:10.400809 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:10.401385 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:10.401389 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:10.401548 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:10.402178 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:25:10.402194 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:25:10.403776 (+ 1582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:25:10.405903 (+ 2127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:10.405907 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:10.405918 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:10.406164 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:10.406496 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:10.406498 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:10.406500 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:10.406624 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:10.407121 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:25:10.407126 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:25:10.407340 (+ 214us) client_negotiation.cc:769] Sending connection context
0808 18:25:10.407547 (+ 207us) client_negotiation.cc:240] Negotiation successful
0808 18:25:10.407787 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":135,"threads_started":1}
I20260808 18:25:10.410269 6018 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.72.189" port: 36153 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 18:25:10 dist-test-slave-5k54 krb5kdc[5856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213510, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.189@KRBTEST.COM for kudu/127.23.72.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.188.keytab.
Entry for principal kudu/127.23.72.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.188.keytab.
I20260808 18:25:10.419016 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.188:36963
--webserver_interface=127.23.72.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.188.keytab
--principal=kudu/127.23.72.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:35725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:33023,127.23.72.189:36153,127.23.72.188:36963
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:25:10.419909 6021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:10.410740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.189:36153 (local address 127.0.0.1:35920)
0808 18:25:10.410919 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:10.410931 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:10.410992 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:10.411610 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:10.411613 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:10.411736 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:25:10.411817 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:10.411822 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:10.412809 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:10.412811 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:10.413369 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:10.413377 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:10.414093 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:10.414096 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:10.414172 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:10.414569 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:25:10.414587 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:25:10.415961 (+ 1374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:25:10.418039 (+ 2078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:10.418045 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:10.418050 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:10.418302 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:10.418651 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:10.418654 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:10.418655 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:10.418739 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:10.419290 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:25:10.419296 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:25:10.419407 (+ 111us) client_negotiation.cc:769] Sending connection context
0808 18:25:10.419524 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 18:25:10.419673 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 18:25:10.420686 6023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:10.410869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35920 (local address 127.23.72.189:36153)
0808 18:25:10.411229 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:10.411237 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:10.411252 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:25:10.411280 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:10.411284 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:10.411468 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:10.411627 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:25:10.411919 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:10.412700 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:10.413494 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:10.413956 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:10.414057 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:10.416077 (+ 2020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:25:10.416085 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:25:10.416094 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:25:10.416114 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:25:10.417918 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:10.418421 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:10.418427 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:10.418428 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:10.418528 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:10.418842 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:10.418845 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:10.418847 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:10.419171 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:25:10.420125 (+ 954us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:10.420238 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 18:25:10.420403 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":181,"threads_started":1}
I20260808 18:25:10.420815 5941 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.72.189" port: 36153 } attempt: 3
I20260808 18:25:10.422765 6018 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.72.188" port: 36963 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:25:10.423504 6021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:10.423241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:36963 (local address 127.0.0.1:33816)
0808 18:25:10.423341 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:10.423413 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.188:36963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 18:25:10.423720 5965 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.72.188:36963: connect: Connection refused (error 111)
I20260808 18:25:10.424680 6021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:10.424465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:36963 (local address 127.0.0.1:33820)
0808 18:25:10.424567 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:10.424616 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.188:36963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 18:25:10.424961 6018 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.72.188:36963: Network error: Client connection negotiation failed: client connection to 127.23.72.188:36963: connect: Connection refused (error 111)
Aug 08 18:25:10 dist-test-slave-5k54 krb5kdc[5856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213510, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for kudu/127.23.72.189@KRBTEST.COM
I20260808 18:25:10.430671 6023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:10.421320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35934 (local address 127.23.72.189:36153)
0808 18:25:10.421674 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:10.421679 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:10.421693 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:25:10.421724 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:10.421726 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:10.421763 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:10.421836 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:25:10.422325 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:10.423038 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:10.424928 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:10.425247 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:10.425327 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:10.427564 (+ 2237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:25:10.427567 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:25:10.427570 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:25:10.427593 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:25:10.429042 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:10.429496 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:10.429500 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:10.429501 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:10.429569 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:10.429858 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:10.429862 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:10.429864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:10.430038 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:25:10.430146 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:10.430430 (+ 284us) server_negotiation.cc:299] Negotiation successful
0808 18:25:10.430535 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268}
I20260808 18:25:10.430677 5945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:10.421282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.189:36153 (local address 127.0.0.1:35934)
0808 18:25:10.421445 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:10.421460 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:10.421534 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:10.421901 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:10.421906 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:10.422111 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:25:10.422213 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:10.422219 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:10.423188 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:10.423194 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:10.424801 (+ 1607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:10.424817 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:10.425375 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:10.425381 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:10.425529 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:10.425994 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:25:10.426013 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:25:10.427394 (+ 1381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:25:10.429166 (+ 1772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:10.429169 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:10.429171 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:10.429382 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:10.429664 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:10.429668 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:10.429670 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:10.429759 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:10.430137 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:25:10.430140 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:25:10.430252 (+ 112us) client_negotiation.cc:769] Sending connection context
0808 18:25:10.430377 (+ 125us) client_negotiation.cc:240] Negotiation successful
0808 18:25:10.430523 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260808 18:25:10.431726 5941 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.72.188" port: 36963 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:25:10.432451 5945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:10.432229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:36963 (local address 127.0.0.1:33824)
0808 18:25:10.432339 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:10.432396 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.188:36963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 18:25:10.433144 5945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:10.432909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:36963 (local address 127.0.0.1:33836)
0808 18:25:10.433046 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:10.433095 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.188:36963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260808 18:25:10.433364 5941 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.72.188:36963: Network error: Client connection negotiation failed: client connection to 127.23.72.188:36963: connect: Connection refused (error 111)
I20260808 18:25:10.464574 5941 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.72.188" port: 36963 } attempt: 1
I20260808 18:25:10.465454 5945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:10.465139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:36963 (local address 127.0.0.1:33848)
0808 18:25:10.465289 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:10.465365 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.188:36963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260808 18:25:10.466203 5945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:10.465969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:36963 (local address 127.0.0.1:33864)
0808 18:25:10.466075 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:10.466124 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.188:36963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 18:25:10.466354 5941 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.72.188:36963: Network error: Client connection negotiation failed: client connection to 127.23.72.188:36963: connect: Connection refused (error 111)
I20260808 18:25:10.474264 6018 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.72.188" port: 36963 } attempt: 1
I20260808 18:25:10.475008 6021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:10.474706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:36963 (local address 127.0.0.1:33866)
0808 18:25:10.474883 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:10.474941 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.188:36963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260808 18:25:10.475766 6021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:10.475549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:36963 (local address 127.0.0.1:33870)
0808 18:25:10.475654 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:10.475701 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.188:36963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 18:25:10.475939 6018 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.72.188:36963: Network error: Client connection negotiation failed: client connection to 127.23.72.188:36963: connect: Connection refused (error 111)
W20260808 18:25:10.540791 6024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:25:10.541114 6024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:25:10.541184 6024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 18:25:10.541507 5941 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.72.188" port: 36963 } attempt: 2
I20260808 18:25:10.542337 5945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:10.542024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:36963 (local address 127.0.0.1:33876)
0808 18:25:10.542193 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:10.542263 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.188:36963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260808 18:25:10.543032 5945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:10.542832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:36963 (local address 127.0.0.1:33878)
0808 18:25:10.542934 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:10.542983 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.188:36963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 18:25:10.543191 5941 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.72.188:36963: Network error: Client connection negotiation failed: client connection to 127.23.72.188:36963: connect: Connection refused (error 111)
I20260808 18:25:10.544133 6018 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.72.188" port: 36963 } attempt: 2
I20260808 18:25:10.544819 6021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:10.544559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:36963 (local address 127.0.0.1:33890)
0808 18:25:10.544680 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:10.544728 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.188:36963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 18:25:10.545470 6021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:10.545263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:36963 (local address 127.0.0.1:33898)
0808 18:25:10.545364 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:10.545418 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.188:36963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 18:25:10.545608 6018 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.72.188:36963: Network error: Client connection negotiation failed: client connection to 127.23.72.188:36963: connect: Connection refused (error 111)
W20260808 18:25:10.545820 6024 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:25:10.545861 6024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:25:10.545881 6024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:25:10.545935 6024 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:25:10.545970 6024 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:25:10.551195 6024 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:35725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:33023,127.23.72.189:36153,127.23.72.188:36963
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.188.keytab
--principal=kudu/127.23.72.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.188:36963
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.23.72.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:25:10.552207 6024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:25:10.553233 6024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:25:10.561563 6034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:10.561580 6032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:10.561605 6031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:25:10.561731 6024 server_base.cc:1034] running on GCE node
I20260808 18:25:10.562639 6024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:25:10.563745 6024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:25:10.564926 6024 hybrid_clock.cc:648] HybridClock initialized: now 1786213510564913 us; error 25 us; skew 500 ppm
Aug 08 18:25:10 dist-test-slave-5k54 krb5kdc[5856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213510, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:25:10.567982 6024 init.cc:377] Logged in from keytab as kudu/127.23.72.188@KRBTEST.COM (short username kudu)
I20260808 18:25:10.568761 6024 webserver.cc:460] Webserver started at http://127.23.72.188:42787/ using document root <none> and password file <none>
I20260808 18:25:10.569257 6024 fs_manager.cc:359] Metadata directory not provided
I20260808 18:25:10.569312 6024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:25:10.569468 6024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:25:10.571412 6024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-2/data/instance:
uuid: "5cd636515de44b5ba582f2f4d6748836"
format_stamp: "Formatted at 2026-08-08 18:25:10 on dist-test-slave-5k54"
server_key: "0af3e55ce0c60e387fc9a25804d99111"
server_key_iv: "f39802722448876000f17d42e807978b"
server_key_version: "encryptionkey@0"
I20260808 18:25:10.572054 6024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-2/wal/instance:
uuid: "5cd636515de44b5ba582f2f4d6748836"
format_stamp: "Formatted at 2026-08-08 18:25:10 on dist-test-slave-5k54"
server_key: "0af3e55ce0c60e387fc9a25804d99111"
server_key_iv: "f39802722448876000f17d42e807978b"
server_key_version: "encryptionkey@0"
I20260808 18:25:10.575817 6024 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20260808 18:25:10.578191 6042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:10.579654 6024 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:25:10.579764 6024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-2/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-2/wal
uuid: "5cd636515de44b5ba582f2f4d6748836"
format_stamp: "Formatted at 2026-08-08 18:25:10 on dist-test-slave-5k54"
server_key: "0af3e55ce0c60e387fc9a25804d99111"
server_key_iv: "f39802722448876000f17d42e807978b"
server_key_version: "encryptionkey@0"
I20260808 18:25:10.579846 6024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:25:10.598229 6024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:25:10.601514 6024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:25:10.601666 6024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:25:10.608708 6024 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.188:36963
I20260808 18:25:10.608726 6096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.188:36963 every 8 connection(s)
I20260808 18:25:10.609731 6024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-2/data/info.pb
I20260808 18:25:10.612457 6097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:25:10.616209 6097 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.72.190" port: 33023 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:25:10.617208 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 6024
I20260808 18:25:10.617359 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213345431840-23842-0/minicluster-data/master-2/wal/instance
I20260808 18:25:10.617659 23842 external_mini_cluster.cc:1428] Setting key 20d9cf76caec241255e388722ef3bb3b
Aug 08 18:25:10 dist-test-slave-5k54 krb5kdc[5856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213510, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.188@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
Aug 08 18:25:10 dist-test-slave-5k54 krb5kdc[5856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.72.188@KRBTEST.COM
I20260808 18:25:10.630664 5944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:10.618362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57266 (local address 127.23.72.190:33023)
0808 18:25:10.618511 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:10.618515 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:10.619340 (+ 825us) server_negotiation.cc:407] Connection header received
0808 18:25:10.620203 (+ 863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:10.620207 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:10.620258 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:10.620355 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:25:10.621566 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:10.622059 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:10.623182 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:10.623392 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:10.623572 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:10.626355 (+ 2783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:25:10.626358 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:25:10.626360 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:25:10.626387 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:25:10.628288 (+ 1901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:10.628851 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:10.628855 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:10.628856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:10.628956 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:10.629404 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:10.629408 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:10.629409 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:10.629656 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:25:10.629753 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:10.630302 (+ 549us) server_negotiation.cc:299] Negotiation successful
0808 18:25:10.630467 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 18:25:10.631655 6100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:10.618697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:33023 (local address 127.0.0.1:57266)
0808 18:25:10.619177 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:10.619225 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:10.619985 (+ 760us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:10.620780 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:10.620784 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:10.621174 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:25:10.621403 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:10.621416 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:10.622174 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:10.622178 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:10.623022 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:10.623031 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:10.623630 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:10.623634 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:10.623771 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:10.624588 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:25:10.624604 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:25:10.626165 (+ 1561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:25:10.628407 (+ 2242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:10.628415 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:10.628423 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:10.628741 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:10.629075 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:10.629081 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:10.629083 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:10.629261 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:10.629766 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:25:10.629773 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:25:10.630021 (+ 248us) client_negotiation.cc:769] Sending connection context
0808 18:25:10.630872 (+ 851us) client_negotiation.cc:240] Negotiation successful
0808 18:25:10.631057 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"mutex_wait_us":48,"thread_start_us":157,"threads_started":1}
I20260808 18:25:10.631656 6101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:10.618698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33900 (local address 127.23.72.188:36963)
0808 18:25:10.619138 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:10.619153 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:10.619184 (+ 31us) server_negotiation.cc:407] Connection header received
0808 18:25:10.619974 (+ 790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:10.619991 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:10.620221 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:10.620400 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:25:10.621118 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:10.622285 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:10.623126 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:10.623621 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:10.623771 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:10.626865 (+ 3094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:25:10.626875 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:25:10.626921 (+ 46us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:25:10.626953 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:25:10.628850 (+ 1897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:10.629393 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:10.629399 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:10.629401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:10.629487 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:10.629792 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:10.629795 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:10.629797 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:10.630054 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:25:10.630249 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:10.630509 (+ 260us) server_negotiation.cc:299] Negotiation successful
0808 18:25:10.630795 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":82,"server-negotiator.queue_time_us":307,"spinlock_wait_cycles":768,"thread_start_us":169,"threads_started":1}
I20260808 18:25:10.633005 6097 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.72.189" port: 36153 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:25:10.636804 6018 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.72.188" port: 36963 } attempt: 3
Aug 08 18:25:10 dist-test-slave-5k54 krb5kdc[5856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213510, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.188@KRBTEST.COM for kudu/127.23.72.189@KRBTEST.COM
Aug 08 18:25:10 dist-test-slave-5k54 krb5kdc[5856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
Aug 08 18:25:10 dist-test-slave-5k54 krb5kdc[5856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213510, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.189@KRBTEST.COM for kudu/127.23.72.188@KRBTEST.COM
I20260808 18:25:10.642350 5941 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.72.188" port: 36963 } attempt: 3
I20260808 18:25:10.643929 6100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:10.633518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.189:36153 (local address 127.0.0.1:35938)
0808 18:25:10.633682 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:10.633697 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:10.633770 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:10.634140 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:10.634143 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:10.634322 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:25:10.634426 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:10.634432 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:10.635457 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:10.635461 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:10.636353 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:10.636362 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:10.636994 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:10.636998 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:10.637139 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:10.637623 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:25:10.637657 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:25:10.639154 (+ 1497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:25:10.642144 (+ 2990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:10.642148 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:10.642150 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:10.642407 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:10.642716 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:10.642719 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:10.642721 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:10.642843 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:10.643375 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:25:10.643379 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:25:10.643519 (+ 140us) client_negotiation.cc:769] Sending connection context
0808 18:25:10.643643 (+ 124us) client_negotiation.cc:240] Negotiation successful
0808 18:25:10.643766 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97}
I20260808 18:25:10.643952 6023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:10.633631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35938 (local address 127.23.72.189:36153)
0808 18:25:10.633784 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:10.633790 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:10.633805 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:25:10.633857 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:10.633860 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:10.633899 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:10.634004 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:25:10.634538 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:10.635308 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:10.636456 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:10.636808 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:10.636910 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:10.640088 (+ 3178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:25:10.640091 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:25:10.640093 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:25:10.640116 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:25:10.642018 (+ 1902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:10.642525 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:10.642529 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:10.642531 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:10.642610 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:10.642961 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:10.642964 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:10.642966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:10.643201 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:25:10.643286 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:10.643700 (+ 414us) server_negotiation.cc:299] Negotiation successful
0808 18:25:10.643806 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 18:25:10.644620 6097 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.72.188" port: 36963 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:25:10.646010 5944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:10.634149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57270 (local address 127.23.72.190:33023)
0808 18:25:10.634276 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:10.634279 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:10.634433 (+ 154us) server_negotiation.cc:407] Connection header received
0808 18:25:10.634582 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:10.634585 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:10.634634 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:10.634693 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:25:10.635457 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:10.636200 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:10.637269 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:10.637615 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:10.637713 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:10.640627 (+ 2914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:25:10.640630 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:25:10.640633 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:25:10.640659 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:25:10.643169 (+ 2510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:10.644171 (+ 1002us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:10.644176 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:10.644178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:10.644264 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:10.644577 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:10.644582 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:10.644583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:10.644825 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:25:10.645067 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:10.645700 (+ 633us) server_negotiation.cc:299] Negotiation successful
0808 18:25:10.645831 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 18:25:10.646679 6021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:10.637219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:36963 (local address 127.0.0.1:33914)
0808 18:25:10.637418 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:10.637430 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:10.637520 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:10.637958 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:10.637961 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:10.638160 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:25:10.638258 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:10.638264 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:10.639277 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:10.639282 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:10.639963 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:10.639972 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:10.640558 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:10.640567 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:10.640655 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:10.641089 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:25:10.641107 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:25:10.642552 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:25:10.645034 (+ 2482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:10.645038 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:10.645040 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:10.645285 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:10.645715 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:10.645719 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:10.645721 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:10.645821 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:10.646227 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:25:10.646232 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:25:10.646327 (+ 95us) client_negotiation.cc:769] Sending connection context
0808 18:25:10.646415 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 18:25:10.646510 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109}
I20260808 18:25:10.647289 6101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:10.637599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33914 (local address 127.23.72.188:36963)
0808 18:25:10.637733 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:10.637738 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:10.637751 (+ 13us) server_negotiation.cc:407] Connection header received
0808 18:25:10.637784 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:10.637787 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:10.637827 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:10.637914 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:25:10.638408 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:10.639154 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:10.640084 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:10.640456 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:10.640530 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:10.642688 (+ 2158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:25:10.642692 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:25:10.642695 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:25:10.642718 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:25:10.644932 (+ 2214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:10.645406 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:10.645410 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:10.645412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:10.645494 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:10.645911 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:10.645914 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:10.645916 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:10.646103 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:25:10.646954 (+ 851us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:10.647035 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 18:25:10.647143 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 18:25:10.650033 6018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d59fd403b144375a6c82b46e0bfcc33: Bootstrap starting.
Aug 08 18:25:10 dist-test-slave-5k54 krb5kdc[5856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.72.189@KRBTEST.COM
I20260808 18:25:10.652930 6018 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d59fd403b144375a6c82b46e0bfcc33: Neither blocks nor log segments found. Creating new log.
I20260808 18:25:10.653795 6018 log.cc:824] T 00000000000000000000000000000000 P 4d59fd403b144375a6c82b46e0bfcc33: Log is configured to *not* fsync() on all Append() calls
I20260808 18:25:10.655857 6023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:10.647341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35950 (local address 127.23.72.189:36153)
0808 18:25:10.647725 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:10.647728 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:10.647738 (+ 10us) server_negotiation.cc:407] Connection header received
0808 18:25:10.647763 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:10.647764 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:10.647794 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:10.647881 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:25:10.648513 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:10.648970 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:10.649702 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:10.649921 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:10.650036 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:10.652610 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:25:10.652613 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:25:10.652616 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:25:10.652646 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:25:10.654060 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:10.654545 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:10.654549 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:10.654551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:10.654637 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:10.654962 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:10.654965 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:10.654967 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:10.655186 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:25:10.655379 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:10.655609 (+ 230us) server_negotiation.cc:299] Negotiation successful
0808 18:25:10.655715 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317}
I20260808 18:25:10.656107 6018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d59fd403b144375a6c82b46e0bfcc33: No bootstrap required, opened a new log
Aug 08 18:25:10 dist-test-slave-5k54 krb5kdc[5856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213510, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.188@KRBTEST.COM for kudu/127.23.72.188@KRBTEST.COM
Aug 08 18:25:10 dist-test-slave-5k54 krb5kdc[5856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213510, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for kudu/127.23.72.188@KRBTEST.COM
I20260808 18:25:10.659730 23842 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:25:10.660699 6018 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d59fd403b144375a6c82b46e0bfcc33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8eea685e974e0b820d809c987df684" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 33023 } } peers { permanent_uuid: "4d59fd403b144375a6c82b46e0bfcc33" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 36153 } } peers { permanent_uuid: "5cd636515de44b5ba582f2f4d6748836" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 36963 } }
I20260808 18:25:10.660964 6018 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4d59fd403b144375a6c82b46e0bfcc33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:25:10.661043 6018 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d59fd403b144375a6c82b46e0bfcc33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d59fd403b144375a6c82b46e0bfcc33, State: Initialized, Role: FOLLOWER
I20260808 18:25:10.661581 6018 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d59fd403b144375a6c82b46e0bfcc33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8eea685e974e0b820d809c987df684" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 33023 } } peers { permanent_uuid: "4d59fd403b144375a6c82b46e0bfcc33" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 36153 } } peers { permanent_uuid: "5cd636515de44b5ba582f2f4d6748836" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 36963 } }
I20260808 18:25:10.662132 6102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:10.645275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33932 (local address 127.23.72.188:36963)
0808 18:25:10.646264 (+ 989us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:10.646274 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:10.646287 (+ 13us) server_negotiation.cc:407] Connection header received
0808 18:25:10.646603 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:10.646606 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:10.646656 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:10.647162 (+ 506us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:25:10.647275 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:10.648035 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:10.655122 (+ 7087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:10.655479 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:10.657707 (+ 2228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:10.657799 (+ 92us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:25:10.657801 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:25:10.657804 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:25:10.657819 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:25:10.660163 (+ 2344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:10.660697 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:10.660700 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:10.660701 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:10.660798 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:10.661123 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:10.661127 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:10.661128 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:10.661385 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:25:10.661474 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:10.661800 (+ 326us) server_negotiation.cc:299] Negotiation successful
0808 18:25:10.661941 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":914,"thread_start_us":109,"threads_started":1}
I20260808 18:25:10.662314 6100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:10.645063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:36963 (local address 127.0.0.1:33932)
0808 18:25:10.645180 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:10.645194 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:10.645261 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:10.646791 (+ 1530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:10.646795 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:10.646966 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:25:10.647048 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:10.647055 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:10.648147 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:10.648151 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:10.654991 (+ 6840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:10.655005 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:10.655598 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:10.655602 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:10.655671 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:10.656226 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:25:10.656240 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:25:10.657589 (+ 1349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:25:10.660301 (+ 2712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:10.660306 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:10.660309 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:10.660575 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:10.660922 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:10.660925 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:10.660928 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:10.661029 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:10.661512 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:25:10.661515 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:25:10.661607 (+ 92us) client_negotiation.cc:769] Sending connection context
0808 18:25:10.661720 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 18:25:10.661849 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":6096}
I20260808 18:25:10.663059 6018 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d59fd403b144375a6c82b46e0bfcc33 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:25:10.665757 6097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cd636515de44b5ba582f2f4d6748836: Bootstrap starting.
I20260808 18:25:10.666004 5945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:10.642714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:36963 (local address 127.0.0.1:33920)
0808 18:25:10.642811 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:10.642826 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:10.642914 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:10.645851 (+ 2937us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:10.645855 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:10.646006 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:25:10.646104 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:10.646111 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:10.656124 (+ 10013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:10.656129 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:10.656918 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:10.656932 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:10.657925 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:10.657929 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:10.658016 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:10.658398 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:25:10.658409 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:25:10.659797 (+ 1388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:25:10.664345 (+ 4548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:10.664349 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:10.664351 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:10.664607 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:10.664975 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:10.664978 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:10.664981 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:10.665089 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:10.665509 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:25:10.665513 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:25:10.665610 (+ 97us) client_negotiation.cc:769] Sending connection context
0808 18:25:10.665701 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 18:25:10.665847 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 18:25:10.666563 6103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:10.643221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33920 (local address 127.23.72.188:36963)
0808 18:25:10.645589 (+ 2368us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:10.645603 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:10.645618 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:25:10.645650 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:10.645653 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:10.645717 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:10.646426 (+ 709us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:25:10.646529 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:10.655972 (+ 9443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:10.657077 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:10.657767 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:10.658204 (+ 437us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:10.661891 (+ 3687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:25:10.661895 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:25:10.661898 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:25:10.661920 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:25:10.664214 (+ 2294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:10.664735 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:10.664741 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:10.664746 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:10.664863 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:10.665203 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:10.665206 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:10.665208 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:10.665395 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:25:10.666238 (+ 843us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:10.666312 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 18:25:10.666419 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":59,"server-negotiator.queue_time_us":2302,"thread_start_us":143,"threads_started":1}
I20260808 18:25:10.669116 6097 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5cd636515de44b5ba582f2f4d6748836: Neither blocks nor log segments found. Creating new log.
I20260808 18:25:10.669425 5941 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf8eea685e974e0b820d809c987df684: Bootstrap starting.
I20260808 18:25:10.669968 6097 log.cc:824] T 00000000000000000000000000000000 P 5cd636515de44b5ba582f2f4d6748836: Log is configured to *not* fsync() on all Append() calls
I20260808 18:25:10.666823 5948 master_runner.cc:186] Error getting master registration for 127.23.72.190:33023: OK, instance_id { permanent_uuid: "bf8eea685e974e0b820d809c987df684" instance_seqno: 1786213510178134 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 18:25:10.671173 5948 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 18:25:10.671499 6111 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d59fd403b144375a6c82b46e0bfcc33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8eea685e974e0b820d809c987df684" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 33023 } } peers { permanent_uuid: "4d59fd403b144375a6c82b46e0bfcc33" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 36153 } } peers { permanent_uuid: "5cd636515de44b5ba582f2f4d6748836" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 36963 } } }
I20260808 18:25:10.671646 6111 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d59fd403b144375a6c82b46e0bfcc33 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:25:10.671993 5941 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf8eea685e974e0b820d809c987df684: Neither blocks nor log segments found. Creating new log.
I20260808 18:25:10.672871 5941 log.cc:824] T 00000000000000000000000000000000 P bf8eea685e974e0b820d809c987df684: Log is configured to *not* fsync() on all Append() calls
I20260808 18:25:10.673697 6102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:10.662723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33940 (local address 127.23.72.188:36963)
0808 18:25:10.662815 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:10.662818 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:10.663740 (+ 922us) server_negotiation.cc:407] Connection header received
0808 18:25:10.663957 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:10.663963 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:10.664019 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:10.664076 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:25:10.665175 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:10.665893 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:10.667703 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:10.668046 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:10.668120 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:10.670032 (+ 1912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:25:10.670037 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:25:10.670041 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:25:10.670064 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:25:10.671662 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:10.672168 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:10.672173 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:10.672176 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:10.672270 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:10.672610 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:10.672613 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:10.672615 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:10.672839 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:25:10.672953 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:10.673396 (+ 443us) server_negotiation.cc:299] Negotiation successful
0808 18:25:10.673519 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 18:25:10.675132 6097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cd636515de44b5ba582f2f4d6748836: No bootstrap required, opened a new log
W20260808 18:25:10.676357 6124 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4d59fd403b144375a6c82b46e0bfcc33: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:25:10.676429 6124 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:25:10.678328 5941 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf8eea685e974e0b820d809c987df684: No bootstrap required, opened a new log
I20260808 18:25:10.680099 6097 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5cd636515de44b5ba582f2f4d6748836 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8eea685e974e0b820d809c987df684" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 33023 } } peers { permanent_uuid: "4d59fd403b144375a6c82b46e0bfcc33" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 36153 } } peers { permanent_uuid: "5cd636515de44b5ba582f2f4d6748836" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 36963 } }
I20260808 18:25:10.680335 6097 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5cd636515de44b5ba582f2f4d6748836 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:25:10.680401 6097 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5cd636515de44b5ba582f2f4d6748836 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd636515de44b5ba582f2f4d6748836, State: Initialized, Role: FOLLOWER
I20260808 18:25:10.680928 6097 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5cd636515de44b5ba582f2f4d6748836 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8eea685e974e0b820d809c987df684" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 33023 } } peers { permanent_uuid: "4d59fd403b144375a6c82b46e0bfcc33" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 36153 } } peers { permanent_uuid: "5cd636515de44b5ba582f2f4d6748836" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 36963 } }
I20260808 18:25:10.682153 6023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:10.672315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35966 (local address 127.23.72.189:36153)
0808 18:25:10.672426 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:10.672430 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:10.672654 (+ 224us) server_negotiation.cc:407] Connection header received
0808 18:25:10.672871 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:10.672875 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:10.672924 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:10.672976 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:25:10.673808 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:10.674494 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:10.676597 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:10.676910 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:10.677015 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:10.678690 (+ 1675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:25:10.678693 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:25:10.678696 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:25:10.678721 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:25:10.680362 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:10.680831 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:10.680835 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:10.680836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:10.680917 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:10.681201 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:10.681204 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:10.681206 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:10.681430 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:25:10.681533 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:10.681881 (+ 348us) server_negotiation.cc:299] Negotiation successful
0808 18:25:10.681997 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 18:25:10.682451 6097 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5cd636515de44b5ba582f2f4d6748836 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:25:10.682745 5941 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf8eea685e974e0b820d809c987df684 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8eea685e974e0b820d809c987df684" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 33023 } } peers { permanent_uuid: "4d59fd403b144375a6c82b46e0bfcc33" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 36153 } } peers { permanent_uuid: "5cd636515de44b5ba582f2f4d6748836" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 36963 } }
I20260808 18:25:10.682968 5941 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf8eea685e974e0b820d809c987df684 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:25:10.683029 5941 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf8eea685e974e0b820d809c987df684 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf8eea685e974e0b820d809c987df684, State: Initialized, Role: FOLLOWER
I20260808 18:25:10.683543 5941 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf8eea685e974e0b820d809c987df684 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8eea685e974e0b820d809c987df684" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 33023 } } peers { permanent_uuid: "4d59fd403b144375a6c82b46e0bfcc33" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 36153 } } peers { permanent_uuid: "5cd636515de44b5ba582f2f4d6748836" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 36963 } }
I20260808 18:25:10.684202 6138 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf8eea685e974e0b820d809c987df684 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8eea685e974e0b820d809c987df684" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 33023 } } peers { permanent_uuid: "4d59fd403b144375a6c82b46e0bfcc33" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 36153 } } peers { permanent_uuid: "5cd636515de44b5ba582f2f4d6748836" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 36963 } } }
I20260808 18:25:10.684389 6138 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf8eea685e974e0b820d809c987df684 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:25:10.685019 5944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:10.672898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57284 (local address 127.23.72.190:33023)
0808 18:25:10.673006 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:10.673010 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:10.674382 (+ 1372us) server_negotiation.cc:407] Connection header received
0808 18:25:10.674549 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:10.674552 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:10.674594 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:10.674640 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:25:10.675597 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:10.676246 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:10.677383 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:10.677988 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:10.678069 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:10.680304 (+ 2235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:25:10.680309 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:25:10.680312 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:25:10.680352 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:25:10.682250 (+ 1898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:10.683317 (+ 1067us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:10.683323 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:10.683326 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:10.683444 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:10.683848 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:10.683852 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:10.683853 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:10.684126 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:25:10.684228 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:10.684697 (+ 469us) server_negotiation.cc:299] Negotiation successful
0808 18:25:10.684860 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 18:25:10.685135 5941 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf8eea685e974e0b820d809c987df684 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:25:10.686723 6132 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cd636515de44b5ba582f2f4d6748836 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8eea685e974e0b820d809c987df684" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 33023 } } peers { permanent_uuid: "4d59fd403b144375a6c82b46e0bfcc33" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 36153 } } peers { permanent_uuid: "5cd636515de44b5ba582f2f4d6748836" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 36963 } } }
I20260808 18:25:10.686852 6132 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cd636515de44b5ba582f2f4d6748836 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 18:25:10.688484 6144 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5cd636515de44b5ba582f2f4d6748836: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:25:10.688606 6144 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:25:10.686529 6024 master_runner.cc:186] Error getting master registration for 127.23.72.190:33023: OK, instance_id { permanent_uuid: "bf8eea685e974e0b820d809c987df684" instance_seqno: 1786213510178134 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 18:25:10.689881 6024 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 18:25:10.690116 6156 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bf8eea685e974e0b820d809c987df684: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:25:10.690188 6156 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:25:10.694106 5869 master_runner.cc:263] No leader master found from master bf8eea685e974e0b820d809c987df684
I20260808 18:25:10.694173 5869 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 18:25:10.707656 6138 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf8eea685e974e0b820d809c987df684 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:25:10.707844 6138 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf8eea685e974e0b820d809c987df684 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8eea685e974e0b820d809c987df684" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 33023 } } peers { permanent_uuid: "4d59fd403b144375a6c82b46e0bfcc33" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 36153 } } peers { permanent_uuid: "5cd636515de44b5ba582f2f4d6748836" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 36963 } }
I20260808 18:25:10.708832 6138 leader_election.cc:288] T 00000000000000000000000000000000 P bf8eea685e974e0b820d809c987df684 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d59fd403b144375a6c82b46e0bfcc33 (127.23.72.189:36153), 5cd636515de44b5ba582f2f4d6748836 (127.23.72.188:36963)
I20260808 18:25:10.709439 6071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bf8eea685e974e0b820d809c987df684" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd636515de44b5ba582f2f4d6748836" is_pre_election: true
I20260808 18:25:10.709728 6071 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5cd636515de44b5ba582f2f4d6748836 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf8eea685e974e0b820d809c987df684 in term 0.
I20260808 18:25:10.710219 5992 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bf8eea685e974e0b820d809c987df684" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d59fd403b144375a6c82b46e0bfcc33" is_pre_election: true
I20260808 18:25:10.710470 5992 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4d59fd403b144375a6c82b46e0bfcc33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf8eea685e974e0b820d809c987df684 in term 0.
I20260808 18:25:10.711701 5885 leader_election.cc:302] T 00000000000000000000000000000000 P bf8eea685e974e0b820d809c987df684 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd636515de44b5ba582f2f4d6748836, bf8eea685e974e0b820d809c987df684; no voters:
I20260808 18:25:10.712149 6138 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf8eea685e974e0b820d809c987df684 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:25:10.712234 6138 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf8eea685e974e0b820d809c987df684 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:25:10.712287 6138 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf8eea685e974e0b820d809c987df684 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:25:10.713747 6138 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf8eea685e974e0b820d809c987df684 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8eea685e974e0b820d809c987df684" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 33023 } } peers { permanent_uuid: "4d59fd403b144375a6c82b46e0bfcc33" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 36153 } } peers { permanent_uuid: "5cd636515de44b5ba582f2f4d6748836" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 36963 } }
I20260808 18:25:10.714082 6138 leader_election.cc:288] T 00000000000000000000000000000000 P bf8eea685e974e0b820d809c987df684 [CANDIDATE]: Term 1 election: Requested vote from peers 4d59fd403b144375a6c82b46e0bfcc33 (127.23.72.189:36153), 5cd636515de44b5ba582f2f4d6748836 (127.23.72.188:36963)
I20260808 18:25:10.714468 5992 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bf8eea685e974e0b820d809c987df684" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d59fd403b144375a6c82b46e0bfcc33"
I20260808 18:25:10.714583 5992 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d59fd403b144375a6c82b46e0bfcc33 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:25:10.716898 6071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bf8eea685e974e0b820d809c987df684" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd636515de44b5ba582f2f4d6748836"
I20260808 18:25:10.717047 6071 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5cd636515de44b5ba582f2f4d6748836 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:25:10.718312 6071 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5cd636515de44b5ba582f2f4d6748836 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf8eea685e974e0b820d809c987df684 in term 1.
I20260808 18:25:10.718568 5885 leader_election.cc:302] T 00000000000000000000000000000000 P bf8eea685e974e0b820d809c987df684 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd636515de44b5ba582f2f4d6748836, bf8eea685e974e0b820d809c987df684; no voters:
I20260808 18:25:10.718725 6138 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf8eea685e974e0b820d809c987df684 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:25:10.718964 6138 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf8eea685e974e0b820d809c987df684 [term 1 LEADER]: Becoming Leader. State: Replica: bf8eea685e974e0b820d809c987df684, State: Running, Role: LEADER
I20260808 18:25:10.719223 5992 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4d59fd403b144375a6c82b46e0bfcc33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf8eea685e974e0b820d809c987df684 in term 1.
I20260808 18:25:10.719338 6138 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf8eea685e974e0b820d809c987df684 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8eea685e974e0b820d809c987df684" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 33023 } } peers { permanent_uuid: "4d59fd403b144375a6c82b46e0bfcc33" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 36153 } } peers { permanent_uuid: "5cd636515de44b5ba582f2f4d6748836" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 36963 } }
I20260808 18:25:10.721163 6138 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf8eea685e974e0b820d809c987df684 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf8eea685e974e0b820d809c987df684. Latest consensus state: current_term: 1 leader_uuid: "bf8eea685e974e0b820d809c987df684" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8eea685e974e0b820d809c987df684" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 33023 } } peers { permanent_uuid: "4d59fd403b144375a6c82b46e0bfcc33" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 36153 } } peers { permanent_uuid: "5cd636515de44b5ba582f2f4d6748836" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 36963 } } }
I20260808 18:25:10.721268 6138 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf8eea685e974e0b820d809c987df684 [sys.catalog]: This master's current role is: LEADER
I20260808 18:25:10.721640 6160 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:25:10.722568 6160 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:25:10.729434 6071 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5cd636515de44b5ba582f2f4d6748836 [term 1 FOLLOWER]: Refusing update from remote peer bf8eea685e974e0b820d809c987df684: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:25:10.730268 5992 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4d59fd403b144375a6c82b46e0bfcc33 [term 1 FOLLOWER]: Refusing update from remote peer bf8eea685e974e0b820d809c987df684: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:25:10.731580 6138 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bf8eea685e974e0b820d809c987df684 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d59fd403b144375a6c82b46e0bfcc33" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:25:10.731968 6138 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bf8eea685e974e0b820d809c987df684 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd636515de44b5ba582f2f4d6748836" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 36963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:25:10.738824 6132 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cd636515de44b5ba582f2f4d6748836 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf8eea685e974e0b820d809c987df684. Latest consensus state: current_term: 1 leader_uuid: "bf8eea685e974e0b820d809c987df684" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8eea685e974e0b820d809c987df684" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 33023 } } peers { permanent_uuid: "4d59fd403b144375a6c82b46e0bfcc33" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 36153 } } peers { permanent_uuid: "5cd636515de44b5ba582f2f4d6748836" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 36963 } } }
I20260808 18:25:10.738951 6132 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cd636515de44b5ba582f2f4d6748836 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:25:10.739271 6111 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d59fd403b144375a6c82b46e0bfcc33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf8eea685e974e0b820d809c987df684. Latest consensus state: current_term: 1 leader_uuid: "bf8eea685e974e0b820d809c987df684" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8eea685e974e0b820d809c987df684" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 33023 } } peers { permanent_uuid: "4d59fd403b144375a6c82b46e0bfcc33" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 36153 } } peers { permanent_uuid: "5cd636515de44b5ba582f2f4d6748836" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 36963 } } }
I20260808 18:25:10.739391 6111 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d59fd403b144375a6c82b46e0bfcc33 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:25:10.745748 6138 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf8eea685e974e0b820d809c987df684 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bf8eea685e974e0b820d809c987df684" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8eea685e974e0b820d809c987df684" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 33023 } } peers { permanent_uuid: "4d59fd403b144375a6c82b46e0bfcc33" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 36153 } } peers { permanent_uuid: "5cd636515de44b5ba582f2f4d6748836" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 36963 } } }
I20260808 18:25:10.745896 6138 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf8eea685e974e0b820d809c987df684 [sys.catalog]: This master's current role is: LEADER
I20260808 18:25:10.746376 6138 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf8eea685e974e0b820d809c987df684 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bf8eea685e974e0b820d809c987df684" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8eea685e974e0b820d809c987df684" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 33023 } } peers { permanent_uuid: "4d59fd403b144375a6c82b46e0bfcc33" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 36153 } } peers { permanent_uuid: "5cd636515de44b5ba582f2f4d6748836" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 36963 } } }
I20260808 18:25:10.746490 6138 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf8eea685e974e0b820d809c987df684 [sys.catalog]: This master's current role is: LEADER
I20260808 18:25:10.746618 6132 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cd636515de44b5ba582f2f4d6748836 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bf8eea685e974e0b820d809c987df684" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8eea685e974e0b820d809c987df684" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 33023 } } peers { permanent_uuid: "4d59fd403b144375a6c82b46e0bfcc33" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 36153 } } peers { permanent_uuid: "5cd636515de44b5ba582f2f4d6748836" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 36963 } } }
I20260808 18:25:10.746730 6132 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cd636515de44b5ba582f2f4d6748836 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:25:10.747538 6111 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d59fd403b144375a6c82b46e0bfcc33 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bf8eea685e974e0b820d809c987df684" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8eea685e974e0b820d809c987df684" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 33023 } } peers { permanent_uuid: "4d59fd403b144375a6c82b46e0bfcc33" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 36153 } } peers { permanent_uuid: "5cd636515de44b5ba582f2f4d6748836" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 36963 } } }
I20260808 18:25:10.747659 6111 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d59fd403b144375a6c82b46e0bfcc33 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:25:10.748054 6161 mvcc.cc:205] Tried to move back new op lower bound from 7316330539939201024 to 7316330539909988352. Current Snapshot: MvccSnapshot[applied={T|T < 7316330539939201024}]
I20260808 18:25:10.749421 6160 catalog_manager.cc:1347] Generated new cluster ID: b7828aac34de47f18444bcc6eb316aef
I20260808 18:25:10.749481 6160 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:25:10.750877 6162 mvcc.cc:205] Tried to move back new op lower bound from 7316330539939201024 to 7316330539909988352. Current Snapshot: MvccSnapshot[applied={T|T < 7316330539939201024}]
I20260808 18:25:10.765637 6163 mvcc.cc:205] Tried to move back new op lower bound from 7316330539939201024 to 7316330539909988352. Current Snapshot: MvccSnapshot[applied={T|T < 7316330539939201024}]
I20260808 18:25:10.782554 6160 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:25:10.783903 6160 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:25:10.819116 6160 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bf8eea685e974e0b820d809c987df684: Generated new TSK 0
I20260808 18:25:10.820035 6160 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:25:11.677910 6124 catalog_manager.cc:1259] Loaded cluster ID: b7828aac34de47f18444bcc6eb316aef
I20260808 18:25:11.678006 6124 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4d59fd403b144375a6c82b46e0bfcc33: loading cluster ID for follower catalog manager: success
I20260808 18:25:11.679682 6124 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4d59fd403b144375a6c82b46e0bfcc33: acquiring CA information for follower catalog manager: success
I20260808 18:25:11.680788 6124 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4d59fd403b144375a6c82b46e0bfcc33: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 18:25:11.689782 6144 catalog_manager.cc:1259] Loaded cluster ID: b7828aac34de47f18444bcc6eb316aef
I20260808 18:25:11.689847 6144 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5cd636515de44b5ba582f2f4d6748836: loading cluster ID for follower catalog manager: success
I20260808 18:25:11.691381 6144 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5cd636515de44b5ba582f2f4d6748836: acquiring CA information for follower catalog manager: success
I20260808 18:25:11.692461 6144 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5cd636515de44b5ba582f2f4d6748836: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 18:25:11.890031 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 5869
I20260808 18:25:11.899163 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 5948
I20260808 18:25:11.905917 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 6024
2026-08-08T18:25:11Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3123 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213345431840-23842-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:25:11 dist-test-slave-5k54 krb5kdc[6177](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:25:11 dist-test-slave-5k54 krb5kdc[6177](info): set up 2 sockets
Aug 08 18:25:11 dist-test-slave-5k54 krb5kdc[6177](info): commencing operation
krb5kdc: starting...
W20260808 18:25:12.979727 23842 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Aug 08 18:25:13 dist-test-slave-5k54 krb5kdc[6177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:25:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:25:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.72.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
I20260808 18:25:13.110359 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:46599
--webserver_interface=127.23.72.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:34093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:46599
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:25:13.220558 6193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:25:13.220853 6193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:25:13.220940 6193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:25:13.224936 6193 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:25:13.225018 6193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:25:13.225055 6193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:25:13.225111 6193 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:25:13.225150 6193 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:25:13.230270 6193 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:34093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:46599
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:46599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.72.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6193
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:25:13.231302 6193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:25:13.232333 6193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:25:13.240857 6200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:13.240864 6198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:25:13.241044 6193 server_base.cc:1034] running on GCE node
W20260808 18:25:13.240875 6202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:25:13.241744 6193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:25:13.242762 6193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:25:13.243922 6193 hybrid_clock.cc:648] HybridClock initialized: now 1786213513243904 us; error 29 us; skew 500 ppm
Aug 08 18:25:13 dist-test-slave-5k54 krb5kdc[6177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213513, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:25:13.246582 6193 init.cc:377] Logged in from keytab as kudu/127.23.72.190@KRBTEST.COM (short username kudu)
I20260808 18:25:13.247483 6193 webserver.cc:460] Webserver started at http://127.23.72.190:38289/ using document root <none> and password file <none>
I20260808 18:25:13.248081 6193 fs_manager.cc:359] Metadata directory not provided
I20260808 18:25:13.248149 6193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:25:13.248312 6193 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:25:13.250237 6193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "555915c4027a4f1899e628e2e7b35f8f"
format_stamp: "Formatted at 2026-08-08 18:25:13 on dist-test-slave-5k54"
server_key: "251bc5d271f5279cbfb1871c56af3b42"
server_key_iv: "30195daec82370b8e9bee82c7feabad5"
server_key_version: "encryptionkey@0"
I20260808 18:25:13.250733 6193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "555915c4027a4f1899e628e2e7b35f8f"
format_stamp: "Formatted at 2026-08-08 18:25:13 on dist-test-slave-5k54"
server_key: "251bc5d271f5279cbfb1871c56af3b42"
server_key_iv: "30195daec82370b8e9bee82c7feabad5"
server_key_version: "encryptionkey@0"
I20260808 18:25:13.254314 6193 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 18:25:13.256644 6209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:13.257845 6193 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:25:13.257982 6193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "555915c4027a4f1899e628e2e7b35f8f"
format_stamp: "Formatted at 2026-08-08 18:25:13 on dist-test-slave-5k54"
server_key: "251bc5d271f5279cbfb1871c56af3b42"
server_key_iv: "30195daec82370b8e9bee82c7feabad5"
server_key_version: "encryptionkey@0"
I20260808 18:25:13.258095 6193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:25:13.272429 6193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:25:13.275246 6193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:25:13.275476 6193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:25:13.282779 6193 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.190:46599
I20260808 18:25:13.282807 6263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.190:46599 every 8 connection(s)
I20260808 18:25:13.283711 6193 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:25:13.286362 6264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:25:13.286892 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 6193
I20260808 18:25:13.287024 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:25:13.287307 23842 external_mini_cluster.cc:1428] Setting key 0f31eff85bdf0db6959bad367c851168
I20260808 18:25:13.292727 6264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 555915c4027a4f1899e628e2e7b35f8f: Bootstrap starting.
I20260808 18:25:13.294906 6264 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 555915c4027a4f1899e628e2e7b35f8f: Neither blocks nor log segments found. Creating new log.
I20260808 18:25:13.295742 6264 log.cc:824] T 00000000000000000000000000000000 P 555915c4027a4f1899e628e2e7b35f8f: Log is configured to *not* fsync() on all Append() calls
Aug 08 18:25:13 dist-test-slave-5k54 krb5kdc[6177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:25:13.297626 6264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 555915c4027a4f1899e628e2e7b35f8f: No bootstrap required, opened a new log
I20260808 18:25:13.301119 6267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:13.288674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39644 (local address 127.23.72.190:46599)
0808 18:25:13.289118 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:13.289132 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:13.289166 (+ 34us) server_negotiation.cc:407] Connection header received
0808 18:25:13.289812 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:13.289828 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:13.290138 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:13.290476 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:25:13.291272 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:13.292324 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:13.293272 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:13.293728 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:13.293893 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:13.296654 (+ 2761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:25:13.296659 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:25:13.296671 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:25:13.296694 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:25:13.298425 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:13.298909 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:13.298913 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:13.298917 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:13.299019 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:13.299296 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:13.299298 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:13.299300 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:13.299721 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:25:13.299871 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:13.300158 (+ 287us) server_negotiation.cc:299] Negotiation successful
0808 18:25:13.300432 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":144,"threads_started":1}
I20260808 18:25:13.301630 6264 raft_consensus.cc:348] T 00000000000000000000000000000000 P 555915c4027a4f1899e628e2e7b35f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "555915c4027a4f1899e628e2e7b35f8f" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 46599 } }
I20260808 18:25:13.301867 6264 raft_consensus.cc:374] T 00000000000000000000000000000000 P 555915c4027a4f1899e628e2e7b35f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:25:13.301947 6264 raft_consensus.cc:729] T 00000000000000000000000000000000 P 555915c4027a4f1899e628e2e7b35f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 555915c4027a4f1899e628e2e7b35f8f, State: Initialized, Role: FOLLOWER
I20260808 18:25:13.302363 6264 consensus_queue.cc:260] T 00000000000000000000000000000000 P 555915c4027a4f1899e628e2e7b35f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "555915c4027a4f1899e628e2e7b35f8f" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 46599 } }
I20260808 18:25:13.302527 6264 raft_consensus.cc:388] T 00000000000000000000000000000000 P 555915c4027a4f1899e628e2e7b35f8f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:25:13.302606 6264 raft_consensus.cc:482] T 00000000000000000000000000000000 P 555915c4027a4f1899e628e2e7b35f8f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:25:13.302728 6264 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 555915c4027a4f1899e628e2e7b35f8f [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:25:13.303781 6264 raft_consensus.cc:504] T 00000000000000000000000000000000 P 555915c4027a4f1899e628e2e7b35f8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "555915c4027a4f1899e628e2e7b35f8f" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 46599 } }
I20260808 18:25:13.303910 23842 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:25:13.304162 6264 leader_election.cc:302] T 00000000000000000000000000000000 P 555915c4027a4f1899e628e2e7b35f8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 555915c4027a4f1899e628e2e7b35f8f; no voters:
I20260808 18:25:13.304459 6264 leader_election.cc:288] T 00000000000000000000000000000000 P 555915c4027a4f1899e628e2e7b35f8f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:25:13.304581 6269 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 555915c4027a4f1899e628e2e7b35f8f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:25:13.304826 6269 raft_consensus.cc:686] T 00000000000000000000000000000000 P 555915c4027a4f1899e628e2e7b35f8f [term 1 LEADER]: Becoming Leader. State: Replica: 555915c4027a4f1899e628e2e7b35f8f, State: Running, Role: LEADER
I20260808 18:25:13.305197 6269 consensus_queue.cc:237] T 00000000000000000000000000000000 P 555915c4027a4f1899e628e2e7b35f8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "555915c4027a4f1899e628e2e7b35f8f" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 46599 } }
I20260808 18:25:13.305764 6264 sys_catalog.cc:564] T 00000000000000000000000000000000 P 555915c4027a4f1899e628e2e7b35f8f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:25:13.306924 6271 sys_catalog.cc:455] T 00000000000000000000000000000000 P 555915c4027a4f1899e628e2e7b35f8f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 555915c4027a4f1899e628e2e7b35f8f. Latest consensus state: current_term: 1 leader_uuid: "555915c4027a4f1899e628e2e7b35f8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "555915c4027a4f1899e628e2e7b35f8f" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 46599 } } }
I20260808 18:25:13.307056 6271 sys_catalog.cc:458] T 00000000000000000000000000000000 P 555915c4027a4f1899e628e2e7b35f8f [sys.catalog]: This master's current role is: LEADER
I20260808 18:25:13.307694 6270 sys_catalog.cc:455] T 00000000000000000000000000000000 P 555915c4027a4f1899e628e2e7b35f8f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "555915c4027a4f1899e628e2e7b35f8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "555915c4027a4f1899e628e2e7b35f8f" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 46599 } } }
I20260808 18:25:13.307817 6270 sys_catalog.cc:458] T 00000000000000000000000000000000 P 555915c4027a4f1899e628e2e7b35f8f [sys.catalog]: This master's current role is: LEADER
I20260808 18:25:13.307714 6279 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:25:13.310370 6279 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:25:13.315765 6279 catalog_manager.cc:1347] Generated new cluster ID: b47846963d3242bcb8618f22aec858a6
I20260808 18:25:13.315840 6279 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:25:13.335948 6279 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:25:13.336923 6279 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:25:13.342348 6279 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 555915c4027a4f1899e628e2e7b35f8f: Generated new TSK 0
I20260808 18:25:13.343047 6279 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:25:13.361147 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 6193
2026-08-08T18:25:13Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1454 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:25:13 dist-test-slave-5k54 krb5kdc[6294](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:25:13 dist-test-slave-5k54 krb5kdc[6294](info): set up 2 sockets
Aug 08 18:25:13 dist-test-slave-5k54 krb5kdc[6294](info): commencing operation
krb5kdc: starting...
W20260808 18:25:14.457217 23842 mini_kdc.cc:121] Time spent starting KDC: real 1.073s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Aug 08 18:25:14 dist-test-slave-5k54 krb5kdc[6294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213514, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:25:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:25:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.72.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
I20260808 18:25:14.592747 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:37911
--webserver_interface=127.23.72.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:38763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:37911
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:25:14.709281 6307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:25:14.709574 6307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:25:14.709643 6307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:25:14.713685 6307 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:25:14.713764 6307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:25:14.713802 6307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:25:14.713853 6307 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:25:14.713884 6307 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:25:14.719537 6307 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:38763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:37911
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:37911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.72.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6307
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:25:14.720580 6307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:25:14.721617 6307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:25:14.730145 6313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:14.730180 6314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:14.730118 6316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:25:14.730487 6307 server_base.cc:1034] running on GCE node
I20260808 18:25:14.730990 6307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:25:14.732126 6307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:25:14.733287 6307 hybrid_clock.cc:648] HybridClock initialized: now 1786213514733274 us; error 26 us; skew 500 ppm
Aug 08 18:25:14 dist-test-slave-5k54 krb5kdc[6294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213514, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:25:14.736090 6307 init.cc:377] Logged in from keytab as kudu/127.23.72.190@KRBTEST.COM (short username kudu)
I20260808 18:25:14.736893 6307 webserver.cc:460] Webserver started at http://127.23.72.190:46233/ using document root <none> and password file <none>
I20260808 18:25:14.737408 6307 fs_manager.cc:359] Metadata directory not provided
I20260808 18:25:14.737465 6307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:25:14.737622 6307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:25:14.739652 6307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "3c4a0053fcaf4116ba005f16ef7094d3"
format_stamp: "Formatted at 2026-08-08 18:25:14 on dist-test-slave-5k54"
server_key: "aeebdb9a77fb4d8b58197855efd3d6d2"
server_key_iv: "53acdc05aa4f45c29539c240224385c6"
server_key_version: "encryptionkey@0"
I20260808 18:25:14.740173 6307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "3c4a0053fcaf4116ba005f16ef7094d3"
format_stamp: "Formatted at 2026-08-08 18:25:14 on dist-test-slave-5k54"
server_key: "aeebdb9a77fb4d8b58197855efd3d6d2"
server_key_iv: "53acdc05aa4f45c29539c240224385c6"
server_key_version: "encryptionkey@0"
I20260808 18:25:14.743798 6307 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 18:25:14.746416 6324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:14.747810 6307 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:25:14.747951 6307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "3c4a0053fcaf4116ba005f16ef7094d3"
format_stamp: "Formatted at 2026-08-08 18:25:14 on dist-test-slave-5k54"
server_key: "aeebdb9a77fb4d8b58197855efd3d6d2"
server_key_iv: "53acdc05aa4f45c29539c240224385c6"
server_key_version: "encryptionkey@0"
I20260808 18:25:14.748063 6307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:25:14.792068 6307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:25:14.795563 6307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:25:14.795766 6307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:25:14.803222 6307 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.190:37911
I20260808 18:25:14.803611 6381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.190:37911 every 8 connection(s)
I20260808 18:25:14.804280 6307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:25:14.807130 6382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:25:14.810057 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 6307
I20260808 18:25:14.810159 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:25:14.810467 23842 external_mini_cluster.cc:1428] Setting key 84c1f1b05dd167a17233527fc5f9fcf8
I20260808 18:25:14.813802 6382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3: Bootstrap starting.
I20260808 18:25:14.816823 6382 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3: Neither blocks nor log segments found. Creating new log.
I20260808 18:25:14.817693 6382 log.cc:824] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3: Log is configured to *not* fsync() on all Append() calls
I20260808 18:25:14.820005 6382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3: No bootstrap required, opened a new log
Aug 08 18:25:14 dist-test-slave-5k54 krb5kdc[6294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213514, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:25:14.824647 6382 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c4a0053fcaf4116ba005f16ef7094d3" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 37911 } }
I20260808 18:25:14.824967 6382 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:25:14.825037 6382 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c4a0053fcaf4116ba005f16ef7094d3, State: Initialized, Role: FOLLOWER
I20260808 18:25:14.825552 6382 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c4a0053fcaf4116ba005f16ef7094d3" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 37911 } }
I20260808 18:25:14.825713 6382 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:25:14.825804 6382 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:25:14.825929 6382 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:25:14.827049 6382 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c4a0053fcaf4116ba005f16ef7094d3" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 37911 } }
I20260808 18:25:14.827471 6382 leader_election.cc:302] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c4a0053fcaf4116ba005f16ef7094d3; no voters:
I20260808 18:25:14.827697 6386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:14.811951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36514 (local address 127.23.72.190:37911)
0808 18:25:14.812393 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:14.812416 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:14.812466 (+ 50us) server_negotiation.cc:407] Connection header received
0808 18:25:14.813367 (+ 901us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:14.813391 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:14.813829 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:14.814233 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:25:14.814923 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:14.816629 (+ 1706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:14.817593 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:14.818349 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:14.818481 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:14.821246 (+ 2765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:25:14.821253 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:25:14.821270 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:25:14.821304 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:25:14.824359 (+ 3055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:14.824857 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:14.824863 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:14.824870 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:14.825030 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:14.825368 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:14.825407 (+ 39us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:14.825408 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:14.825936 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:25:14.826155 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:14.826470 (+ 315us) server_negotiation.cc:299] Negotiation successful
0808 18:25:14.826771 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":100,"threads_started":1}
I20260808 18:25:14.827868 6382 leader_election.cc:288] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:25:14.828011 6388 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:25:14.828230 6388 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [term 1 LEADER]: Becoming Leader. State: Replica: 3c4a0053fcaf4116ba005f16ef7094d3, State: Running, Role: LEADER
I20260808 18:25:14.828563 6388 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c4a0053fcaf4116ba005f16ef7094d3" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 37911 } }
I20260808 18:25:14.829262 6382 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:25:14.830636 6390 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c4a0053fcaf4116ba005f16ef7094d3. Latest consensus state: current_term: 1 leader_uuid: "3c4a0053fcaf4116ba005f16ef7094d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c4a0053fcaf4116ba005f16ef7094d3" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 37911 } } }
I20260808 18:25:14.830765 6390 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [sys.catalog]: This master's current role is: LEADER
I20260808 18:25:14.831627 6397 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:25:14.831079 6389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c4a0053fcaf4116ba005f16ef7094d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c4a0053fcaf4116ba005f16ef7094d3" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 37911 } } }
I20260808 18:25:14.831817 6389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [sys.catalog]: This master's current role is: LEADER
I20260808 18:25:14.834642 6397 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:25:14.840380 6397 catalog_manager.cc:1347] Generated new cluster ID: a4d84cc9953c490ea6a9eeee5225bbea
I20260808 18:25:14.840461 6397 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:25:14.858623 6397 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:25:14.859644 6397 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:25:14.866977 6397 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3: Generated new TSK 0
I20260808 18:25:14.867801 6397 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.72.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab.
Entry for principal kudu/127.23.72.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab.
I20260808 18:25:14.905745 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.129:0
--local_ip_for_outbound_sockets=127.23.72.129
--webserver_interface=127.23.72.129
--webserver_port=0
--tserver_master_addrs=127.23.72.190:37911
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab
--principal=kudu/127.23.72.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:38763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:25:15.018361 6409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:25:15.018616 6409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:25:15.018657 6409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:25:15.022753 6409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:25:15.022810 6409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:25:15.022868 6409 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.129
I20260808 18:25:15.028044 6409 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:38763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab
--principal=kudu/127.23.72.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.72.129
--webserver_port=0
--tserver_master_addrs=127.23.72.190:37911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.129
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:25:15.028982 6409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:25:15.029983 6409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:25:15.039511 6419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:25:15.039553 6409 server_base.cc:1034] running on GCE node
W20260808 18:25:15.039556 6418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:15.039592 6421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:25:15.040127 6409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:25:15.040733 6409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:25:15.041916 6409 hybrid_clock.cc:648] HybridClock initialized: now 1786213515041902 us; error 27 us; skew 500 ppm
Aug 08 18:25:15 dist-test-slave-5k54 krb5kdc[6294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213515, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:25:15.044735 6409 init.cc:377] Logged in from keytab as kudu/127.23.72.129@KRBTEST.COM (short username kudu)
I20260808 18:25:15.045542 6409 webserver.cc:460] Webserver started at http://127.23.72.129:42033/ using document root <none> and password file <none>
I20260808 18:25:15.046032 6409 fs_manager.cc:359] Metadata directory not provided
I20260808 18:25:15.046089 6409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:25:15.046248 6409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:25:15.048234 6409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-0/data/instance:
uuid: "c573d0f081194f1ebaaf440a0edb5074"
format_stamp: "Formatted at 2026-08-08 18:25:15 on dist-test-slave-5k54"
server_key: "f2a7fa5f70a86fdfa0e8196d8a11fafe"
server_key_iv: "7590036b47f987afc856a07d08544ff5"
server_key_version: "encryptionkey@0"
I20260808 18:25:15.048741 6409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance:
uuid: "c573d0f081194f1ebaaf440a0edb5074"
format_stamp: "Formatted at 2026-08-08 18:25:15 on dist-test-slave-5k54"
server_key: "f2a7fa5f70a86fdfa0e8196d8a11fafe"
server_key_iv: "7590036b47f987afc856a07d08544ff5"
server_key_version: "encryptionkey@0"
I20260808 18:25:15.052541 6409 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 18:25:15.054888 6428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:15.056143 6409 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 18:25:15.056245 6409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-0/wal
uuid: "c573d0f081194f1ebaaf440a0edb5074"
format_stamp: "Formatted at 2026-08-08 18:25:15 on dist-test-slave-5k54"
server_key: "f2a7fa5f70a86fdfa0e8196d8a11fafe"
server_key_iv: "7590036b47f987afc856a07d08544ff5"
server_key_version: "encryptionkey@0"
I20260808 18:25:15.056342 6409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:25:15.091339 6409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:25:15.094744 6409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:25:15.094908 6409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:25:15.095512 6409 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:25:15.096304 6409 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:25:15.096356 6409 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:15.096397 6409 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:25:15.096415 6409 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:15.105736 6409 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.129:42583
I20260808 18:25:15.105755 6547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.129:42583 every 8 connection(s)
I20260808 18:25:15.106757 6409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
I20260808 18:25:15.112890 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 6409
I20260808 18:25:15.113056 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance
I20260808 18:25:15.113348 23842 external_mini_cluster.cc:1428] Setting key d88dd0755a8245f58ac23347a03bd0d4
Aug 08 18:25:15 dist-test-slave-5k54 krb5kdc[6294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213515, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:25:15.121380 6386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:15.108812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:34651 (local address 127.23.72.190:37911)
0808 18:25:15.109041 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:15.109044 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:15.109887 (+ 843us) server_negotiation.cc:407] Connection header received
0808 18:25:15.111301 (+ 1414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:15.111304 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:15.111355 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:15.111493 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:25:15.112780 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:15.113505 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:15.114696 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:15.114948 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:15.115064 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:15.117669 (+ 2605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:25:15.117672 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:25:15.117674 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:25:15.117699 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:25:15.119205 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:15.119798 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:15.119802 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:15.119804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:15.119895 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:15.120244 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:15.120247 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:15.120249 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:15.120443 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:25:15.120547 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:15.121060 (+ 513us) server_negotiation.cc:299] Negotiation successful
0808 18:25:15.121199 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131}
I20260808 18:25:15.122239 6550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:15.109212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.129:34651)
0808 18:25:15.109716 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:15.109764 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:15.111080 (+ 1316us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:15.111664 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:15.111672 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:15.112078 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:25:15.112604 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:15.112626 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:15.113641 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:15.113645 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:15.114596 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:15.114606 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:15.115107 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:15.115112 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:15.115289 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:15.115985 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:25:15.116002 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:25:15.117499 (+ 1497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:25:15.119330 (+ 1831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:15.119336 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:15.119361 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:15.119696 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:15.119989 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:15.119992 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:15.119994 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:15.120144 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:15.120547 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:25:15.120555 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:25:15.120813 (+ 258us) client_negotiation.cc:769] Sending connection context
0808 18:25:15.121004 (+ 191us) client_negotiation.cc:240] Negotiation successful
0808 18:25:15.121345 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":133,"threads_started":1}
WARNING: no policy specified for kudu/127.23.72.130@KRBTEST.COM; defaulting to no policy
I20260808 18:25:15.123660 6548 heartbeater.cc:347] Connected to a master server at 127.23.72.190:37911
I20260808 18:25:15.123943 6548 heartbeater.cc:464] Registering TS with master...
I20260808 18:25:15.124469 6548 heartbeater.cc:511] Master 127.23.72.190:37911 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.130@KRBTEST.COM" created.
I20260808 18:25:15.126304 6344 ts_manager.cc:194] Registered new tserver with Master: c573d0f081194f1ebaaf440a0edb5074 (127.23.72.129:42583)
I20260808 18:25:15.127660 6344 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.129@KRBTEST.COM'} at 127.23.72.129:34651
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab.
Entry for principal kudu/127.23.72.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab.
I20260808 18:25:15.143412 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.130:0
--local_ip_for_outbound_sockets=127.23.72.130
--webserver_interface=127.23.72.130
--webserver_port=0
--tserver_master_addrs=127.23.72.190:37911
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab
--principal=kudu/127.23.72.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:38763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:25:15.255748 6553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:25:15.256001 6553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:25:15.256075 6553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:25:15.260172 6553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:25:15.260227 6553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:25:15.260309 6553 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.130
I20260808 18:25:15.265450 6553 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:38763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab
--principal=kudu/127.23.72.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.72.130
--webserver_port=0
--tserver_master_addrs=127.23.72.190:37911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.130
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:25:15.266458 6553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:25:15.267516 6553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:25:15.276670 6559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:15.276702 6561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:15.276676 6558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:25:15.276957 6553 server_base.cc:1034] running on GCE node
I20260808 18:25:15.277446 6553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:25:15.278105 6553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:25:15.279289 6553 hybrid_clock.cc:648] HybridClock initialized: now 1786213515279275 us; error 25 us; skew 500 ppm
Aug 08 18:25:15 dist-test-slave-5k54 krb5kdc[6294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213515, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:25:15.282259 6553 init.cc:377] Logged in from keytab as kudu/127.23.72.130@KRBTEST.COM (short username kudu)
I20260808 18:25:15.283102 6553 webserver.cc:460] Webserver started at http://127.23.72.130:39023/ using document root <none> and password file <none>
I20260808 18:25:15.283715 6553 fs_manager.cc:359] Metadata directory not provided
I20260808 18:25:15.283805 6553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:25:15.284070 6553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:25:15.286103 6553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-1/data/instance:
uuid: "477e6890969f4889a60eea8e71c7201f"
format_stamp: "Formatted at 2026-08-08 18:25:15 on dist-test-slave-5k54"
server_key: "17346c4437bfba79f09bae0e10393e27"
server_key_iv: "a0b37598c0509654609f3e886bcfbe26"
server_key_version: "encryptionkey@0"
I20260808 18:25:15.286679 6553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance:
uuid: "477e6890969f4889a60eea8e71c7201f"
format_stamp: "Formatted at 2026-08-08 18:25:15 on dist-test-slave-5k54"
server_key: "17346c4437bfba79f09bae0e10393e27"
server_key_iv: "a0b37598c0509654609f3e886bcfbe26"
server_key_version: "encryptionkey@0"
I20260808 18:25:15.290400 6553 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 18:25:15.292800 6569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:15.293957 6553 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:25:15.294091 6553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-1/wal
uuid: "477e6890969f4889a60eea8e71c7201f"
format_stamp: "Formatted at 2026-08-08 18:25:15 on dist-test-slave-5k54"
server_key: "17346c4437bfba79f09bae0e10393e27"
server_key_iv: "a0b37598c0509654609f3e886bcfbe26"
server_key_version: "encryptionkey@0"
I20260808 18:25:15.294196 6553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:25:15.309273 6553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:25:15.312498 6553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:25:15.312695 6553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:25:15.313292 6553 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:25:15.314090 6553 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:25:15.314155 6553 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:15.314226 6553 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:25:15.314266 6553 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:15.323093 6553 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.130:38425
I20260808 18:25:15.323122 6688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.130:38425 every 8 connection(s)
I20260808 18:25:15.324002 6553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
I20260808 18:25:15.330469 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 6553
I20260808 18:25:15.330618 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance
I20260808 18:25:15.330895 23842 external_mini_cluster.cc:1428] Setting key 3d1e466e1d959053dab184243a13140d
Aug 08 18:25:15 dist-test-slave-5k54 krb5kdc[6294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213515, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.130@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:25:15.338199 6386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:15.325779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:39805 (local address 127.23.72.190:37911)
0808 18:25:15.325956 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:15.325959 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:15.326738 (+ 779us) server_negotiation.cc:407] Connection header received
0808 18:25:15.328231 (+ 1493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:15.328234 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:15.328299 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:15.328405 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:25:15.329630 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:15.330171 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:15.331859 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:15.332149 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:15.332263 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:15.334707 (+ 2444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:25:15.334711 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:25:15.334714 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:25:15.334745 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:25:15.336133 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:15.336645 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:15.336649 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:15.336651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:15.336731 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:15.337080 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:15.337083 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:15.337085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:15.337270 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:25:15.337380 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:15.337903 (+ 523us) server_negotiation.cc:299] Negotiation successful
0808 18:25:15.338035 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260808 18:25:15.339015 6691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:15.326130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.130:39805)
0808 18:25:15.326576 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:15.326619 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:15.328011 (+ 1392us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:15.328574 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:15.328582 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:15.328961 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:25:15.329488 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:15.329505 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:15.330340 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:15.330343 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:15.331718 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:15.331729 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:15.332315 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:15.332321 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:15.332506 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:15.333104 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:25:15.333120 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:25:15.334543 (+ 1423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:25:15.336251 (+ 1708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:15.336257 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:15.336272 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:15.336542 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:15.336831 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:15.336834 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:15.336836 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:15.336983 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:15.337385 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:25:15.337394 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:25:15.337674 (+ 280us) client_negotiation.cc:769] Sending connection context
0808 18:25:15.337860 (+ 186us) client_negotiation.cc:240] Negotiation successful
0808 18:25:15.338184 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":118,"threads_started":1}
I20260808 18:25:15.340188 6689 heartbeater.cc:347] Connected to a master server at 127.23.72.190:37911
I20260808 18:25:15.340436 6689 heartbeater.cc:464] Registering TS with master...
I20260808 18:25:15.340907 6689 heartbeater.cc:511] Master 127.23.72.190:37911 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.23.72.131@KRBTEST.COM; defaulting to no policy
I20260808 18:25:15.341993 6344 ts_manager.cc:194] Registered new tserver with Master: 477e6890969f4889a60eea8e71c7201f (127.23.72.130:38425)
I20260808 18:25:15.342633 6344 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.130@KRBTEST.COM'} at 127.23.72.130:39805
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab.
Entry for principal kudu/127.23.72.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab.
I20260808 18:25:15.358433 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.131:0
--local_ip_for_outbound_sockets=127.23.72.131
--webserver_interface=127.23.72.131
--webserver_port=0
--tserver_master_addrs=127.23.72.190:37911
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab
--principal=kudu/127.23.72.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:38763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:25:15.472386 6694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:25:15.472635 6694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:25:15.472677 6694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:25:15.476579 6694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:25:15.476630 6694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:25:15.476683 6694 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.131
I20260808 18:25:15.481753 6694 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:38763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab
--principal=kudu/127.23.72.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.72.131
--webserver_port=0
--tserver_master_addrs=127.23.72.190:37911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.131
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:25:15.482703 6694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:25:15.483763 6694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:25:15.493106 6700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:15.493153 6699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:15.493135 6702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:25:15.493407 6694 server_base.cc:1034] running on GCE node
I20260808 18:25:15.493784 6694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:25:15.494463 6694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:25:15.495644 6694 hybrid_clock.cc:648] HybridClock initialized: now 1786213515495626 us; error 28 us; skew 500 ppm
Aug 08 18:25:15 dist-test-slave-5k54 krb5kdc[6294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213515, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:25:15.498414 6694 init.cc:377] Logged in from keytab as kudu/127.23.72.131@KRBTEST.COM (short username kudu)
I20260808 18:25:15.499239 6694 webserver.cc:460] Webserver started at http://127.23.72.131:40977/ using document root <none> and password file <none>
I20260808 18:25:15.499802 6694 fs_manager.cc:359] Metadata directory not provided
I20260808 18:25:15.499869 6694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:25:15.500032 6694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:25:15.502154 6694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-2/data/instance:
uuid: "447f81cd31b74dccbf6a0d3b113a81f4"
format_stamp: "Formatted at 2026-08-08 18:25:15 on dist-test-slave-5k54"
server_key: "a2082b064639115d217cba86fcedb5ea"
server_key_iv: "b6a58bf6c75d31a6cf218a7c810d308a"
server_key_version: "encryptionkey@0"
I20260808 18:25:15.502696 6694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance:
uuid: "447f81cd31b74dccbf6a0d3b113a81f4"
format_stamp: "Formatted at 2026-08-08 18:25:15 on dist-test-slave-5k54"
server_key: "a2082b064639115d217cba86fcedb5ea"
server_key_iv: "b6a58bf6c75d31a6cf218a7c810d308a"
server_key_version: "encryptionkey@0"
I20260808 18:25:15.506331 6694 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 18:25:15.508693 6709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:15.509928 6694 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:25:15.510020 6694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-2/wal
uuid: "447f81cd31b74dccbf6a0d3b113a81f4"
format_stamp: "Formatted at 2026-08-08 18:25:15 on dist-test-slave-5k54"
server_key: "a2082b064639115d217cba86fcedb5ea"
server_key_iv: "b6a58bf6c75d31a6cf218a7c810d308a"
server_key_version: "encryptionkey@0"
I20260808 18:25:15.510098 6694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:25:15.546545 6694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:25:15.549964 6694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:25:15.550127 6694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:25:15.550669 6694 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:25:15.551517 6694 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:25:15.551582 6694 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:15.551625 6694 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:25:15.551646 6694 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:15.561666 6694 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.131:40735
I20260808 18:25:15.561688 6830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.131:40735 every 8 connection(s)
I20260808 18:25:15.562533 6694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
I20260808 18:25:15.565299 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 6694
I20260808 18:25:15.565443 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance
I20260808 18:25:15.565712 23842 external_mini_cluster.cc:1428] Setting key 8822012c6c133b770b5690acd6c79fc0
Aug 08 18:25:15 dist-test-slave-5k54 krb5kdc[6294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213515, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.131@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:25:15.576072 6386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:15.564328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:58781 (local address 127.23.72.190:37911)
0808 18:25:15.564481 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:15.564485 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:15.565246 (+ 761us) server_negotiation.cc:407] Connection header received
0808 18:25:15.566921 (+ 1675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:15.566924 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:15.566983 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:15.567078 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:25:15.568374 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:15.568911 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:15.570031 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:15.570251 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:15.570354 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:15.572781 (+ 2427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:25:15.572789 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:25:15.572792 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:25:15.572815 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:25:15.574022 (+ 1207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:15.574523 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:15.574527 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:15.574528 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:15.574603 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:15.574931 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:15.574935 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:15.574937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:15.575112 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:25:15.575228 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:15.575769 (+ 541us) server_negotiation.cc:299] Negotiation successful
0808 18:25:15.575902 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 18:25:15.576813 6833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:15.564638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.131:58781)
0808 18:25:15.565104 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:15.565147 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:15.566696 (+ 1549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:15.567251 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:15.567258 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:15.567709 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:25:15.568230 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:15.568244 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:15.569034 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:15.569042 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:15.569907 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:15.569915 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:15.570413 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:15.570417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:15.570584 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:15.571084 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:25:15.571100 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:25:15.572625 (+ 1525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:25:15.574141 (+ 1516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:15.574147 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:15.574160 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:15.574413 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:15.574695 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:15.574698 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:15.574700 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:15.574831 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:15.575228 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:25:15.575235 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:25:15.575518 (+ 283us) client_negotiation.cc:769] Sending connection context
0808 18:25:15.575722 (+ 204us) client_negotiation.cc:240] Negotiation successful
0808 18:25:15.576026 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":129,"threads_started":1}
I20260808 18:25:15.578073 6831 heartbeater.cc:347] Connected to a master server at 127.23.72.190:37911
I20260808 18:25:15.578351 6831 heartbeater.cc:464] Registering TS with master...
I20260808 18:25:15.578941 6831 heartbeater.cc:511] Master 127.23.72.190:37911 requested a full tablet report, sending...
I20260808 18:25:15.579983 6344 ts_manager.cc:194] Registered new tserver with Master: 447f81cd31b74dccbf6a0d3b113a81f4 (127.23.72.131:40735)
I20260808 18:25:15.580550 6344 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.131@KRBTEST.COM'} at 127.23.72.131:58781
I20260808 18:25:15.591238 23842 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:25:15.600615 6386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:15.593271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36520 (local address 127.23.72.190:37911)
0808 18:25:15.593386 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:15.593390 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:15.593567 (+ 177us) server_negotiation.cc:407] Connection header received
0808 18:25:15.593678 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:15.593680 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:15.593716 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:15.593821 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:25:15.594481 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:15.595006 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:15.595890 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:15.596097 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:15.596176 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:15.597600 (+ 1424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:25:15.597602 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:25:15.597604 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:25:15.597626 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:25:15.599152 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:15.599560 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:15.599562 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:15.599564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:15.599636 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:15.599873 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:15.599874 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:15.599876 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:15.600071 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:25:15.600208 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:15.600409 (+ 201us) server_negotiation.cc:299] Negotiation successful
0808 18:25:15.600502 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 18:25:15.603552 6344 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36520:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:25:15.606566 6344 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:25:15.621115 6844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:15.615123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.131:40735 (local address 127.0.0.1:56120)
0808 18:25:15.615988 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:15.616004 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:15.616094 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:15.617040 (+ 946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:15.617045 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:15.617066 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:25:15.617154 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:15.617159 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:15.618694 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:15.618701 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:15.619680 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:15.619689 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:15.620725 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:15.620728 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:15.620844 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:15.620865 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 18:25:15.620910 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 18:25:15.620958 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":803,"mutex_wait_us":3,"thread_start_us":98,"threads_started":1}
I20260808 18:25:15.621122 6842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:15.615054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:42583 (local address 127.0.0.1:55758)
0808 18:25:15.615439 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:15.615496 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:15.615622 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:15.616586 (+ 964us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:15.616594 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:15.616623 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:25:15.616773 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:15.616782 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:15.618057 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:15.618062 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:15.619310 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:15.619321 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:15.620509 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:15.620513 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:15.620629 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:15.620689 (+ 60us) client_negotiation.cc:769] Sending connection context
0808 18:25:15.620818 (+ 129us) client_negotiation.cc:240] Negotiation successful
0808 18:25:15.620943 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":34,"thread_start_us":81,"threads_started":1}
I20260808 18:25:15.621330 6843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:15.615398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.130:38425 (local address 127.0.0.1:34650)
0808 18:25:15.616264 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:15.616276 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:15.616363 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:15.616818 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:15.616821 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:15.616851 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:25:15.616974 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:15.616984 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:15.618609 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:15.618612 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:15.619786 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:15.619795 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:15.620843 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:15.620847 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:15.620941 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:15.620964 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 18:25:15.621038 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 18:25:15.621115 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":808,"thread_start_us":105,"threads_started":1}
I20260808 18:25:15.621333 6846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:15.615889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34650 (local address 127.23.72.130:38425)
0808 18:25:15.616338 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:15.616350 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:15.616372 (+ 22us) server_negotiation.cc:407] Connection header received
0808 18:25:15.616476 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:15.616483 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:15.616666 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:15.617127 (+ 461us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:25:15.617291 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:15.618461 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:15.619922 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:15.620720 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:15.620838 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:15.620969 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:15.621080 (+ 111us) server_negotiation.cc:299] Negotiation successful
0808 18:25:15.621177 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":151,"threads_started":1}
I20260808 18:25:15.621765 6845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:15.615771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55758 (local address 127.23.72.129:42583)
0808 18:25:15.616026 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:15.616046 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:15.616070 (+ 24us) server_negotiation.cc:407] Connection header received
0808 18:25:15.616149 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:15.616154 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:15.616394 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:15.616943 (+ 549us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:25:15.617128 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:15.617924 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:15.619477 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:15.620341 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:15.620444 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:15.620594 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:15.621207 (+ 613us) server_negotiation.cc:299] Negotiation successful
0808 18:25:15.621652 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":101,"threads_started":1}
I20260808 18:25:15.623121 6847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:15.616214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56120 (local address 127.23.72.131:40735)
0808 18:25:15.616607 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:15.616623 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:15.616645 (+ 22us) server_negotiation.cc:407] Connection header received
0808 18:25:15.616716 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:15.616724 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:15.616892 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:15.617290 (+ 398us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:25:15.617450 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:15.618526 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:15.619823 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:15.620582 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:15.622639 (+ 2057us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:15.622770 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:15.622856 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 18:25:15.622953 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":134,"threads_started":1}
I20260808 18:25:15.624831 6480 tablet_service.cc:1467] Processing CreateTablet for tablet 94e20ad153154a879f941cc5c215913c (DEFAULT_TABLE table=test-table [id=3f4ab3bf020741c5afca8cbbfa825a45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:25:15.625814 6480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94e20ad153154a879f941cc5c215913c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:25:15.625379 6621 tablet_service.cc:1467] Processing CreateTablet for tablet 94e20ad153154a879f941cc5c215913c (DEFAULT_TABLE table=test-table [id=3f4ab3bf020741c5afca8cbbfa825a45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:25:15.625892 6763 tablet_service.cc:1467] Processing CreateTablet for tablet 94e20ad153154a879f941cc5c215913c (DEFAULT_TABLE table=test-table [id=3f4ab3bf020741c5afca8cbbfa825a45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:25:15.626458 6621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94e20ad153154a879f941cc5c215913c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:25:15.626839 6763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94e20ad153154a879f941cc5c215913c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:25:15.632598 6848 tablet_bootstrap.cc:492] T 94e20ad153154a879f941cc5c215913c P c573d0f081194f1ebaaf440a0edb5074: Bootstrap starting.
I20260808 18:25:15.634423 6851 tablet_bootstrap.cc:492] T 94e20ad153154a879f941cc5c215913c P 477e6890969f4889a60eea8e71c7201f: Bootstrap starting.
I20260808 18:25:15.634969 6848 tablet_bootstrap.cc:654] T 94e20ad153154a879f941cc5c215913c P c573d0f081194f1ebaaf440a0edb5074: Neither blocks nor log segments found. Creating new log.
I20260808 18:25:15.635766 6850 tablet_bootstrap.cc:492] T 94e20ad153154a879f941cc5c215913c P 447f81cd31b74dccbf6a0d3b113a81f4: Bootstrap starting.
I20260808 18:25:15.635866 6848 log.cc:824] T 94e20ad153154a879f941cc5c215913c P c573d0f081194f1ebaaf440a0edb5074: Log is configured to *not* fsync() on all Append() calls
I20260808 18:25:15.636971 6851 tablet_bootstrap.cc:654] T 94e20ad153154a879f941cc5c215913c P 477e6890969f4889a60eea8e71c7201f: Neither blocks nor log segments found. Creating new log.
I20260808 18:25:15.637704 6850 tablet_bootstrap.cc:654] T 94e20ad153154a879f941cc5c215913c P 447f81cd31b74dccbf6a0d3b113a81f4: Neither blocks nor log segments found. Creating new log.
I20260808 18:25:15.637833 6851 log.cc:824] T 94e20ad153154a879f941cc5c215913c P 477e6890969f4889a60eea8e71c7201f: Log is configured to *not* fsync() on all Append() calls
I20260808 18:25:15.638233 6848 tablet_bootstrap.cc:492] T 94e20ad153154a879f941cc5c215913c P c573d0f081194f1ebaaf440a0edb5074: No bootstrap required, opened a new log
I20260808 18:25:15.638430 6848 ts_tablet_manager.cc:1397] T 94e20ad153154a879f941cc5c215913c P c573d0f081194f1ebaaf440a0edb5074: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 18:25:15.638890 6850 log.cc:824] T 94e20ad153154a879f941cc5c215913c P 447f81cd31b74dccbf6a0d3b113a81f4: Log is configured to *not* fsync() on all Append() calls
I20260808 18:25:15.639827 6851 tablet_bootstrap.cc:492] T 94e20ad153154a879f941cc5c215913c P 477e6890969f4889a60eea8e71c7201f: No bootstrap required, opened a new log
I20260808 18:25:15.640035 6851 ts_tablet_manager.cc:1397] T 94e20ad153154a879f941cc5c215913c P 477e6890969f4889a60eea8e71c7201f: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 18:25:15.640566 6850 tablet_bootstrap.cc:492] T 94e20ad153154a879f941cc5c215913c P 447f81cd31b74dccbf6a0d3b113a81f4: No bootstrap required, opened a new log
I20260808 18:25:15.640748 6850 ts_tablet_manager.cc:1397] T 94e20ad153154a879f941cc5c215913c P 447f81cd31b74dccbf6a0d3b113a81f4: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 18:25:15.642917 6848 raft_consensus.cc:348] T 94e20ad153154a879f941cc5c215913c P c573d0f081194f1ebaaf440a0edb5074 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c573d0f081194f1ebaaf440a0edb5074" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 42583 } } peers { permanent_uuid: "447f81cd31b74dccbf6a0d3b113a81f4" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40735 } } peers { permanent_uuid: "477e6890969f4889a60eea8e71c7201f" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 38425 } }
I20260808 18:25:15.643191 6848 raft_consensus.cc:374] T 94e20ad153154a879f941cc5c215913c P c573d0f081194f1ebaaf440a0edb5074 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:25:15.643283 6848 raft_consensus.cc:729] T 94e20ad153154a879f941cc5c215913c P c573d0f081194f1ebaaf440a0edb5074 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c573d0f081194f1ebaaf440a0edb5074, State: Initialized, Role: FOLLOWER
I20260808 18:25:15.643467 6850 raft_consensus.cc:348] T 94e20ad153154a879f941cc5c215913c P 447f81cd31b74dccbf6a0d3b113a81f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c573d0f081194f1ebaaf440a0edb5074" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 42583 } } peers { permanent_uuid: "447f81cd31b74dccbf6a0d3b113a81f4" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40735 } } peers { permanent_uuid: "477e6890969f4889a60eea8e71c7201f" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 38425 } }
I20260808 18:25:15.643733 6850 raft_consensus.cc:374] T 94e20ad153154a879f941cc5c215913c P 447f81cd31b74dccbf6a0d3b113a81f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:25:15.643814 6850 raft_consensus.cc:729] T 94e20ad153154a879f941cc5c215913c P 447f81cd31b74dccbf6a0d3b113a81f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 447f81cd31b74dccbf6a0d3b113a81f4, State: Initialized, Role: FOLLOWER
I20260808 18:25:15.643817 6848 consensus_queue.cc:260] T 94e20ad153154a879f941cc5c215913c P c573d0f081194f1ebaaf440a0edb5074 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c573d0f081194f1ebaaf440a0edb5074" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 42583 } } peers { permanent_uuid: "447f81cd31b74dccbf6a0d3b113a81f4" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40735 } } peers { permanent_uuid: "477e6890969f4889a60eea8e71c7201f" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 38425 } }
I20260808 18:25:15.643761 6851 raft_consensus.cc:348] T 94e20ad153154a879f941cc5c215913c P 477e6890969f4889a60eea8e71c7201f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c573d0f081194f1ebaaf440a0edb5074" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 42583 } } peers { permanent_uuid: "447f81cd31b74dccbf6a0d3b113a81f4" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40735 } } peers { permanent_uuid: "477e6890969f4889a60eea8e71c7201f" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 38425 } }
I20260808 18:25:15.644019 6851 raft_consensus.cc:374] T 94e20ad153154a879f941cc5c215913c P 477e6890969f4889a60eea8e71c7201f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:25:15.644091 6851 raft_consensus.cc:729] T 94e20ad153154a879f941cc5c215913c P 477e6890969f4889a60eea8e71c7201f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 477e6890969f4889a60eea8e71c7201f, State: Initialized, Role: FOLLOWER
I20260808 18:25:15.644276 6850 consensus_queue.cc:260] T 94e20ad153154a879f941cc5c215913c P 447f81cd31b74dccbf6a0d3b113a81f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c573d0f081194f1ebaaf440a0edb5074" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 42583 } } peers { permanent_uuid: "447f81cd31b74dccbf6a0d3b113a81f4" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40735 } } peers { permanent_uuid: "477e6890969f4889a60eea8e71c7201f" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 38425 } }
I20260808 18:25:15.644553 6851 consensus_queue.cc:260] T 94e20ad153154a879f941cc5c215913c P 477e6890969f4889a60eea8e71c7201f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c573d0f081194f1ebaaf440a0edb5074" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 42583 } } peers { permanent_uuid: "447f81cd31b74dccbf6a0d3b113a81f4" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40735 } } peers { permanent_uuid: "477e6890969f4889a60eea8e71c7201f" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 38425 } }
I20260808 18:25:15.644922 6848 ts_tablet_manager.cc:1428] T 94e20ad153154a879f941cc5c215913c P c573d0f081194f1ebaaf440a0edb5074: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 18:25:15.644995 6831 heartbeater.cc:503] Master 127.23.72.190:37911 was elected leader, sending a full tablet report...
I20260808 18:25:15.645248 6548 heartbeater.cc:503] Master 127.23.72.190:37911 was elected leader, sending a full tablet report...
I20260808 18:25:15.645262 6850 ts_tablet_manager.cc:1428] T 94e20ad153154a879f941cc5c215913c P 447f81cd31b74dccbf6a0d3b113a81f4: Time spent starting tablet: real 0.004s user 0.000s sys 0.005s
I20260808 18:25:15.645618 6851 ts_tablet_manager.cc:1428] T 94e20ad153154a879f941cc5c215913c P 477e6890969f4889a60eea8e71c7201f: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 18:25:15.645936 6689 heartbeater.cc:503] Master 127.23.72.190:37911 was elected leader, sending a full tablet report...
W20260808 18:25:15.813474 6832 tablet.cc:2369] T 94e20ad153154a879f941cc5c215913c P 447f81cd31b74dccbf6a0d3b113a81f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 18:25:15.825109 6690 tablet.cc:2369] T 94e20ad153154a879f941cc5c215913c P 477e6890969f4889a60eea8e71c7201f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:25:15.826288 6857 raft_consensus.cc:482] T 94e20ad153154a879f941cc5c215913c P 477e6890969f4889a60eea8e71c7201f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:25:15.826502 6857 raft_consensus.cc:504] T 94e20ad153154a879f941cc5c215913c P 477e6890969f4889a60eea8e71c7201f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c573d0f081194f1ebaaf440a0edb5074" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 42583 } } peers { permanent_uuid: "447f81cd31b74dccbf6a0d3b113a81f4" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40735 } } peers { permanent_uuid: "477e6890969f4889a60eea8e71c7201f" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 38425 } }
I20260808 18:25:15.827739 6857 leader_election.cc:288] T 94e20ad153154a879f941cc5c215913c P 477e6890969f4889a60eea8e71c7201f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c573d0f081194f1ebaaf440a0edb5074 (127.23.72.129:42583), 447f81cd31b74dccbf6a0d3b113a81f4 (127.23.72.131:40735)
I20260808 18:25:15.833005 6847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:15.828155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:50771 (local address 127.23.72.131:40735)
0808 18:25:15.828295 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:15.828299 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:15.828316 (+ 17us) server_negotiation.cc:407] Connection header received
0808 18:25:15.828462 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:15.828464 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:15.828527 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:15.828614 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:25:15.828936 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:15.829742 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:15.831679 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:15.832317 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:15.832451 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:15.832504 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:15.832797 (+ 293us) server_negotiation.cc:299] Negotiation successful
0808 18:25:15.832869 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 18:25:15.833005 6845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:15.828099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:54529 (local address 127.23.72.129:42583)
0808 18:25:15.828227 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:15.828233 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:15.828539 (+ 306us) server_negotiation.cc:407] Connection header received
0808 18:25:15.828605 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:15.828608 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:15.828650 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:15.828722 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:25:15.828967 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:15.829742 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:15.831674 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:15.832320 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:15.832451 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:15.832504 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:15.832797 (+ 293us) server_negotiation.cc:299] Negotiation successful
0808 18:25:15.832869 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 18:25:15.833061 6691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:15.828027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.131:40735 (local address 127.23.72.130:50771)
0808 18:25:15.828227 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:15.828242 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:15.828341 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:15.828640 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:15.828642 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:15.828654 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:25:15.828795 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:15.828802 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:15.829849 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:15.829853 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:15.831518 (+ 1665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:15.831530 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:15.832459 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:15.832463 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:15.832661 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:15.832700 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 18:25:15.832769 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 18:25:15.832897 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":327}
I20260808 18:25:15.833070 6858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:15.828026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:42583 (local address 127.23.72.130:54529)
0808 18:25:15.828440 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:15.828455 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:15.828538 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:15.828737 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:15.828739 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:15.828758 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:25:15.828864 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:15.828872 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:15.829849 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:15.829853 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:15.831520 (+ 1667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:15.831530 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:15.832459 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:15.832463 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:15.832633 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:15.832688 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 18:25:15.832770 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 18:25:15.832871 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"mutex_wait_us":221,"thread_start_us":110,"threads_started":1}
I20260808 18:25:15.833817 6785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94e20ad153154a879f941cc5c215913c" candidate_uuid: "477e6890969f4889a60eea8e71c7201f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "447f81cd31b74dccbf6a0d3b113a81f4" is_pre_election: true
I20260808 18:25:15.833925 6502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94e20ad153154a879f941cc5c215913c" candidate_uuid: "477e6890969f4889a60eea8e71c7201f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c573d0f081194f1ebaaf440a0edb5074" is_pre_election: true
I20260808 18:25:15.834177 6785 raft_consensus.cc:2454] T 94e20ad153154a879f941cc5c215913c P 447f81cd31b74dccbf6a0d3b113a81f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 477e6890969f4889a60eea8e71c7201f in term 0.
I20260808 18:25:15.834220 6502 raft_consensus.cc:2454] T 94e20ad153154a879f941cc5c215913c P c573d0f081194f1ebaaf440a0edb5074 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 477e6890969f4889a60eea8e71c7201f in term 0.
I20260808 18:25:15.834678 6571 leader_election.cc:302] T 94e20ad153154a879f941cc5c215913c P 477e6890969f4889a60eea8e71c7201f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 477e6890969f4889a60eea8e71c7201f, c573d0f081194f1ebaaf440a0edb5074; no voters:
I20260808 18:25:15.834975 6857 raft_consensus.cc:2781] T 94e20ad153154a879f941cc5c215913c P 477e6890969f4889a60eea8e71c7201f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:25:15.835059 6857 raft_consensus.cc:482] T 94e20ad153154a879f941cc5c215913c P 477e6890969f4889a60eea8e71c7201f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:25:15.835107 6857 raft_consensus.cc:3037] T 94e20ad153154a879f941cc5c215913c P 477e6890969f4889a60eea8e71c7201f [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:25:15.836177 6857 raft_consensus.cc:504] T 94e20ad153154a879f941cc5c215913c P 477e6890969f4889a60eea8e71c7201f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c573d0f081194f1ebaaf440a0edb5074" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 42583 } } peers { permanent_uuid: "447f81cd31b74dccbf6a0d3b113a81f4" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40735 } } peers { permanent_uuid: "477e6890969f4889a60eea8e71c7201f" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 38425 } }
I20260808 18:25:15.836562 6857 leader_election.cc:288] T 94e20ad153154a879f941cc5c215913c P 477e6890969f4889a60eea8e71c7201f [CANDIDATE]: Term 1 election: Requested vote from peers c573d0f081194f1ebaaf440a0edb5074 (127.23.72.129:42583), 447f81cd31b74dccbf6a0d3b113a81f4 (127.23.72.131:40735)
I20260808 18:25:15.836876 6502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94e20ad153154a879f941cc5c215913c" candidate_uuid: "477e6890969f4889a60eea8e71c7201f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c573d0f081194f1ebaaf440a0edb5074"
I20260808 18:25:15.836939 6785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94e20ad153154a879f941cc5c215913c" candidate_uuid: "477e6890969f4889a60eea8e71c7201f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "447f81cd31b74dccbf6a0d3b113a81f4"
I20260808 18:25:15.837013 6502 raft_consensus.cc:3037] T 94e20ad153154a879f941cc5c215913c P c573d0f081194f1ebaaf440a0edb5074 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:25:15.837064 6785 raft_consensus.cc:3037] T 94e20ad153154a879f941cc5c215913c P 447f81cd31b74dccbf6a0d3b113a81f4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:25:15.838057 6785 raft_consensus.cc:2454] T 94e20ad153154a879f941cc5c215913c P 447f81cd31b74dccbf6a0d3b113a81f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 477e6890969f4889a60eea8e71c7201f in term 1.
I20260808 18:25:15.838078 6502 raft_consensus.cc:2454] T 94e20ad153154a879f941cc5c215913c P c573d0f081194f1ebaaf440a0edb5074 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 477e6890969f4889a60eea8e71c7201f in term 1.
I20260808 18:25:15.838342 6572 leader_election.cc:302] T 94e20ad153154a879f941cc5c215913c P 477e6890969f4889a60eea8e71c7201f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 447f81cd31b74dccbf6a0d3b113a81f4, 477e6890969f4889a60eea8e71c7201f; no voters:
I20260808 18:25:15.838543 6857 raft_consensus.cc:2781] T 94e20ad153154a879f941cc5c215913c P 477e6890969f4889a60eea8e71c7201f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:25:15.838745 6857 raft_consensus.cc:686] T 94e20ad153154a879f941cc5c215913c P 477e6890969f4889a60eea8e71c7201f [term 1 LEADER]: Becoming Leader. State: Replica: 477e6890969f4889a60eea8e71c7201f, State: Running, Role: LEADER
I20260808 18:25:15.839066 6857 consensus_queue.cc:237] T 94e20ad153154a879f941cc5c215913c P 477e6890969f4889a60eea8e71c7201f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c573d0f081194f1ebaaf440a0edb5074" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 42583 } } peers { permanent_uuid: "447f81cd31b74dccbf6a0d3b113a81f4" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40735 } } peers { permanent_uuid: "477e6890969f4889a60eea8e71c7201f" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 38425 } }
I20260808 18:25:15.842356 6344 catalog_manager.cc:5495] T 94e20ad153154a879f941cc5c215913c P 477e6890969f4889a60eea8e71c7201f reported cstate change: term changed from 0 to 1, leader changed from <none> to 477e6890969f4889a60eea8e71c7201f (127.23.72.130). New cstate: current_term: 1 leader_uuid: "477e6890969f4889a60eea8e71c7201f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c573d0f081194f1ebaaf440a0edb5074" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 42583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "447f81cd31b74dccbf6a0d3b113a81f4" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40735 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "477e6890969f4889a60eea8e71c7201f" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 38425 } health_report { overall_health: HEALTHY } } }
I20260808 18:25:15.854233 6846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:15.850774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34660 (local address 127.23.72.130:38425)
0808 18:25:15.850880 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:15.850883 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:15.850911 (+ 28us) server_negotiation.cc:407] Connection header received
0808 18:25:15.851082 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:15.851085 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:15.851136 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:15.851217 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:25:15.851497 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:15.852017 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:15.852912 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:15.853118 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:15.853188 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:15.853445 (+ 257us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:25:15.853832 (+ 387us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:25:15.853921 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:15.853979 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 18:25:15.854047 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
W20260808 18:25:15.858275 6549 tablet.cc:2369] T 94e20ad153154a879f941cc5c215913c P c573d0f081194f1ebaaf440a0edb5074: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:25:15.860052 6502 raft_consensus.cc:1261] T 94e20ad153154a879f941cc5c215913c P c573d0f081194f1ebaaf440a0edb5074 [term 1 FOLLOWER]: Refusing update from remote peer 477e6890969f4889a60eea8e71c7201f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:25:15.860056 6785 raft_consensus.cc:1261] T 94e20ad153154a879f941cc5c215913c P 447f81cd31b74dccbf6a0d3b113a81f4 [term 1 FOLLOWER]: Refusing update from remote peer 477e6890969f4889a60eea8e71c7201f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:25:15.860739 6859 consensus_queue.cc:1035] T 94e20ad153154a879f941cc5c215913c P 477e6890969f4889a60eea8e71c7201f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c573d0f081194f1ebaaf440a0edb5074" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 42583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:25:15.860952 6857 consensus_queue.cc:1035] T 94e20ad153154a879f941cc5c215913c P 477e6890969f4889a60eea8e71c7201f [LEADER]: Connected to new peer: Peer: permanent_uuid: "447f81cd31b74dccbf6a0d3b113a81f4" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40735 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:25:15.870198 6863 mvcc.cc:205] Tried to move back new op lower bound from 7316330560954855424 to 7316330560879706112. Current Snapshot: MvccSnapshot[applied={T|T < 7316330560954855424 or (T in {7316330560954855424})}]
I20260808 18:25:15.871044 6864 mvcc.cc:205] Tried to move back new op lower bound from 7316330560954855424 to 7316330560879706112. Current Snapshot: MvccSnapshot[applied={T|T < 7316330560954855424}]
I20260808 18:25:15.875927 6339 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36520:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:25:15.876065 6339 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36520:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:25:15.878366 6339 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3: Sending DeleteTablet for 3 replicas of tablet 94e20ad153154a879f941cc5c215913c
I20260808 18:25:15.879156 6480 tablet_service.cc:1514] Processing DeleteTablet for tablet 94e20ad153154a879f941cc5c215913c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:25:15 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:55758
I20260808 18:25:15.879163 6763 tablet_service.cc:1514] Processing DeleteTablet for tablet 94e20ad153154a879f941cc5c215913c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:25:15 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:56120
I20260808 18:25:15.879256 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 6409
I20260808 18:25:15.881623 6621 tablet_service.cc:1514] Processing DeleteTablet for tablet 94e20ad153154a879f941cc5c215913c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:25:15 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:34650
I20260808 18:25:15.881888 6871 tablet_replica.cc:330] T 94e20ad153154a879f941cc5c215913c P 447f81cd31b74dccbf6a0d3b113a81f4: stopping tablet replica
I20260808 18:25:15.882148 6872 tablet_replica.cc:330] T 94e20ad153154a879f941cc5c215913c P 477e6890969f4889a60eea8e71c7201f: stopping tablet replica
I20260808 18:25:15.882300 6871 raft_consensus.cc:2229] T 94e20ad153154a879f941cc5c215913c P 447f81cd31b74dccbf6a0d3b113a81f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:25:15.882472 6872 raft_consensus.cc:2229] T 94e20ad153154a879f941cc5c215913c P 477e6890969f4889a60eea8e71c7201f [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:25:15.882700 6871 raft_consensus.cc:2258] T 94e20ad153154a879f941cc5c215913c P 447f81cd31b74dccbf6a0d3b113a81f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:25:15.882836 6872 raft_consensus.cc:2258] T 94e20ad153154a879f941cc5c215913c P 477e6890969f4889a60eea8e71c7201f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:25:15.884248 6872 ts_tablet_manager.cc:1905] T 94e20ad153154a879f941cc5c215913c P 477e6890969f4889a60eea8e71c7201f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:25:15.887068 6872 ts_tablet_manager.cc:1918] T 94e20ad153154a879f941cc5c215913c P 477e6890969f4889a60eea8e71c7201f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:25:15.887164 6872 log.cc:1196] T 94e20ad153154a879f941cc5c215913c P 477e6890969f4889a60eea8e71c7201f: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-1/wal/wals/94e20ad153154a879f941cc5c215913c
I20260808 18:25:15.887631 6872 ts_tablet_manager.cc:1939] T 94e20ad153154a879f941cc5c215913c P 477e6890969f4889a60eea8e71c7201f: Deleting consensus metadata
I20260808 18:25:15.888737 6871 ts_tablet_manager.cc:1905] T 94e20ad153154a879f941cc5c215913c P 447f81cd31b74dccbf6a0d3b113a81f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:25:15.888835 6329 catalog_manager.cc:4841] TS 477e6890969f4889a60eea8e71c7201f (127.23.72.130:38425): tablet 94e20ad153154a879f941cc5c215913c (table test-table [id=3f4ab3bf020741c5afca8cbbfa825a45]) successfully deleted
W20260808 18:25:15.889225 6327 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.72.129:42583 (error 108)
I20260808 18:25:15.890223 6843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:15.889937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:42583 (local address 127.0.0.1:55772)
0808 18:25:15.890048 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:15.890125 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.129:42583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 18:25:15.890364 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 6553
W20260808 18:25:15.890435 6327 catalog_manager.cc:4584] TS c573d0f081194f1ebaaf440a0edb5074 (127.23.72.129:42583): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 94e20ad153154a879f941cc5c215913c: Network error: Client connection negotiation failed: client connection to 127.23.72.129:42583: connect: Connection refused (error 111)
I20260808 18:25:15.894534 6871 ts_tablet_manager.cc:1918] T 94e20ad153154a879f941cc5c215913c P 447f81cd31b74dccbf6a0d3b113a81f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:25:15.894647 6871 log.cc:1196] T 94e20ad153154a879f941cc5c215913c P 447f81cd31b74dccbf6a0d3b113a81f4: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-2/wal/wals/94e20ad153154a879f941cc5c215913c
I20260808 18:25:15.894944 6871 ts_tablet_manager.cc:1939] T 94e20ad153154a879f941cc5c215913c P 447f81cd31b74dccbf6a0d3b113a81f4: Deleting consensus metadata
I20260808 18:25:15.896057 6328 catalog_manager.cc:4841] TS 447f81cd31b74dccbf6a0d3b113a81f4 (127.23.72.131:40735): tablet 94e20ad153154a879f941cc5c215913c (table test-table [id=3f4ab3bf020741c5afca8cbbfa825a45]) successfully deleted
I20260808 18:25:15.900051 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 6694
I20260808 18:25:15.906781 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 6307
I20260808 18:25:15.914705 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:37911
--webserver_interface=127.23.72.190
--webserver_port=46233
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:38763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:37911
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:25:16.025813 6874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:25:16.026073 6874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:25:16.026113 6874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:25:16.030066 6874 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:25:16.030120 6874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:25:16.030139 6874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:25:16.030155 6874 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:25:16.030171 6874 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:25:16.035331 6874 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:38763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:37911
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:37911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.72.190
--webserver_port=46233
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6874
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:25:16.036405 6874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:25:16.037441 6874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:25:16.051570 6884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:16.051580 6881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:16.051580 6882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:25:16.052006 6874 server_base.cc:1034] running on GCE node
I20260808 18:25:16.052636 6874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:25:16.053761 6874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:25:16.054942 6874 hybrid_clock.cc:648] HybridClock initialized: now 1786213516054921 us; error 36 us; skew 500 ppm
Aug 08 18:25:16 dist-test-slave-5k54 krb5kdc[6294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213516, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:25:16.057857 6874 init.cc:377] Logged in from keytab as kudu/127.23.72.190@KRBTEST.COM (short username kudu)
I20260808 18:25:16.058777 6874 webserver.cc:460] Webserver started at http://127.23.72.190:46233/ using document root <none> and password file <none>
I20260808 18:25:16.059353 6874 fs_manager.cc:359] Metadata directory not provided
I20260808 18:25:16.059499 6874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:25:16.063354 6874 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260808 18:25:16.065596 6891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:16.066898 6874 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 18:25:16.067049 6874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "3c4a0053fcaf4116ba005f16ef7094d3"
format_stamp: "Formatted at 2026-08-08 18:25:14 on dist-test-slave-5k54"
server_key: "aeebdb9a77fb4d8b58197855efd3d6d2"
server_key_iv: "53acdc05aa4f45c29539c240224385c6"
server_key_version: "encryptionkey@0"
I20260808 18:25:16.067731 6874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:25:16.084671 6874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:25:16.087560 6874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:25:16.087759 6874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:25:16.095113 6874 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.190:37911
I20260808 18:25:16.095486 6947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.190:37911 every 8 connection(s)
I20260808 18:25:16.096344 6874 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:25:16.100725 6948 sys_catalog.cc:263] Verifying existing consensus state
I20260808 18:25:16.101368 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 6874
I20260808 18:25:16.101814 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.129:42583
--local_ip_for_outbound_sockets=127.23.72.129
--tserver_master_addrs=127.23.72.190:37911
--webserver_port=42033
--webserver_interface=127.23.72.129
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab
--principal=kudu/127.23.72.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:38763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:25:16.102530 6948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3: Bootstrap starting.
I20260808 18:25:16.120522 6948 log.cc:824] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3: Log is configured to *not* fsync() on all Append() calls
I20260808 18:25:16.129595 6948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 18:25:16.129984 6948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3: Bootstrap complete.
I20260808 18:25:16.134109 6948 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c4a0053fcaf4116ba005f16ef7094d3" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 37911 } }
I20260808 18:25:16.134498 6948 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c4a0053fcaf4116ba005f16ef7094d3, State: Initialized, Role: FOLLOWER
I20260808 18:25:16.134968 6948 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c4a0053fcaf4116ba005f16ef7094d3" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 37911 } }
I20260808 18:25:16.135082 6948 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:25:16.135170 6948 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:25:16.135284 6948 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [term 1 FOLLOWER]: Advancing to term 2
I20260808 18:25:16.137262 6948 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c4a0053fcaf4116ba005f16ef7094d3" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 37911 } }
I20260808 18:25:16.137602 6948 leader_election.cc:302] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c4a0053fcaf4116ba005f16ef7094d3; no voters:
I20260808 18:25:16.137928 6948 leader_election.cc:288] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 18:25:16.138013 6952 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 18:25:16.138209 6952 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [term 2 LEADER]: Becoming Leader. State: Replica: 3c4a0053fcaf4116ba005f16ef7094d3, State: Running, Role: LEADER
I20260808 18:25:16.138517 6952 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c4a0053fcaf4116ba005f16ef7094d3" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 37911 } }
I20260808 18:25:16.139170 6948 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:25:16.139515 6952 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c4a0053fcaf4116ba005f16ef7094d3. Latest consensus state: current_term: 2 leader_uuid: "3c4a0053fcaf4116ba005f16ef7094d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c4a0053fcaf4116ba005f16ef7094d3" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 37911 } } }
I20260808 18:25:16.139639 6952 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [sys.catalog]: This master's current role is: LEADER
I20260808 18:25:16.140051 6953 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "3c4a0053fcaf4116ba005f16ef7094d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c4a0053fcaf4116ba005f16ef7094d3" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 37911 } } }
I20260808 18:25:16.140220 6953 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [sys.catalog]: This master's current role is: LEADER
I20260808 18:25:16.141320 6956 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:25:16.146231 6956 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:25:16.146893 6956 catalog_manager.cc:1259] Loaded cluster ID: a4d84cc9953c490ea6a9eeee5225bbea
I20260808 18:25:16.146953 6956 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:25:16.148353 6956 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260808 18:25:16.153012 6956 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260808 18:25:16.158248 6956 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:25:16.158993 6956 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3: Loaded TSK: 0
I20260808 18:25:16.160270 6956 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 18:25:16.222396 6950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:25:16.222638 6950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:25:16.222681 6950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:25:16.226747 6950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:25:16.226811 6950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:25:16.226868 6950 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.129
I20260808 18:25:16.232146 6950 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:38763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab
--principal=kudu/127.23.72.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.129:42583
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.72.129
--webserver_port=42033
--tserver_master_addrs=127.23.72.190:37911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6950
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.129
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:25:16.233152 6950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:25:16.234166 6950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:25:16.243781 6976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:16.243901 6975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:16.243747 6978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:25:16.244305 6950 server_base.cc:1034] running on GCE node
I20260808 18:25:16.244753 6950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:25:16.245451 6950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:25:16.246629 6950 hybrid_clock.cc:648] HybridClock initialized: now 1786213516246600 us; error 43 us; skew 500 ppm
Aug 08 18:25:16 dist-test-slave-5k54 krb5kdc[6294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213516, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:25:16.249855 6950 init.cc:377] Logged in from keytab as kudu/127.23.72.129@KRBTEST.COM (short username kudu)
I20260808 18:25:16.250847 6950 webserver.cc:460] Webserver started at http://127.23.72.129:42033/ using document root <none> and password file <none>
I20260808 18:25:16.251412 6950 fs_manager.cc:359] Metadata directory not provided
I20260808 18:25:16.251554 6950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:25:16.255558 6950 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260808 18:25:16.257740 6985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:16.259094 6950 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:25:16.259204 6950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-0/wal
uuid: "c573d0f081194f1ebaaf440a0edb5074"
format_stamp: "Formatted at 2026-08-08 18:25:15 on dist-test-slave-5k54"
server_key: "f2a7fa5f70a86fdfa0e8196d8a11fafe"
server_key_iv: "7590036b47f987afc856a07d08544ff5"
server_key_version: "encryptionkey@0"
I20260808 18:25:16.259816 6950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:25:16.276707 6950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:25:16.279908 6950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:25:16.280110 6950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:25:16.280758 6950 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:25:16.281837 6992 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 18:25:16.285955 6950 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 18:25:16.286015 6950 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260808 18:25:16.286083 6950 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 18:25:16.287750 6950 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 18:25:16.287796 6950 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.002s
I20260808 18:25:16.288058 6992 tablet_bootstrap.cc:492] T 94e20ad153154a879f941cc5c215913c P c573d0f081194f1ebaaf440a0edb5074: Bootstrap starting.
I20260808 18:25:16.298560 6950 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.129:42583
I20260808 18:25:16.298655 7105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.129:42583 every 8 connection(s)
I20260808 18:25:16.299612 6950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
I20260808 18:25:16.306514 6992 log.cc:824] T 94e20ad153154a879f941cc5c215913c P c573d0f081194f1ebaaf440a0edb5074: Log is configured to *not* fsync() on all Append() calls
I20260808 18:25:16.309541 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 6950
I20260808 18:25:16.310075 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.130:38425
--local_ip_for_outbound_sockets=127.23.72.130
--tserver_master_addrs=127.23.72.190:37911
--webserver_port=39023
--webserver_interface=127.23.72.130
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab
--principal=kudu/127.23.72.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:38763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 18:25:16 dist-test-slave-5k54 krb5kdc[6294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213516, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:25:16.315265 6992 tablet_bootstrap.cc:492] T 94e20ad153154a879f941cc5c215913c P c573d0f081194f1ebaaf440a0edb5074: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 18:25:16.315696 6992 tablet_bootstrap.cc:492] T 94e20ad153154a879f941cc5c215913c P c573d0f081194f1ebaaf440a0edb5074: Bootstrap complete.
I20260808 18:25:16.316864 6992 ts_tablet_manager.cc:1397] T 94e20ad153154a879f941cc5c215913c P c573d0f081194f1ebaaf440a0edb5074: Time spent bootstrapping tablet: real 0.029s user 0.017s sys 0.010s
I20260808 18:25:16.317687 7109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:16.301623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:39379 (local address 127.23.72.190:37911)
0808 18:25:16.302144 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:16.302163 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:16.302213 (+ 50us) server_negotiation.cc:407] Connection header received
0808 18:25:16.303383 (+ 1170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:16.303400 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:16.303846 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:16.304305 (+ 459us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:25:16.305336 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:16.306792 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:16.308202 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:16.308877 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:16.309057 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:16.311585 (+ 2528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:25:16.311590 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:25:16.311601 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:25:16.311630 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:25:16.313733 (+ 2103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:16.314250 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:16.314256 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:16.314261 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:16.314417 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:16.314854 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:16.314857 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:16.314859 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:16.315301 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:25:16.315470 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:16.316245 (+ 775us) server_negotiation.cc:299] Negotiation successful
0808 18:25:16.316619 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":132,"threads_started":1}
I20260808 18:25:16.319353 7108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:16.301572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.129:39379)
0808 18:25:16.302062 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:16.302118 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:16.302934 (+ 816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:16.304317 (+ 1383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:16.304324 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:16.304673 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:25:16.305152 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:16.305167 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:16.306980 (+ 1813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:16.306983 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:16.308068 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:16.308079 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:16.309005 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:16.309008 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:16.309145 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:16.309805 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:25:16.309821 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:25:16.311354 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:25:16.313854 (+ 2500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:16.313859 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:16.313869 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:16.314139 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:16.314535 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:16.314541 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:16.314545 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:16.314736 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:16.315507 (+ 771us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:25:16.315517 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:25:16.315858 (+ 341us) client_negotiation.cc:769] Sending connection context
0808 18:25:16.317985 (+ 2127us) client_negotiation.cc:240] Negotiation successful
0808 18:25:16.318351 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":136,"threads_started":1}
I20260808 18:25:16.319226 6992 raft_consensus.cc:348] T 94e20ad153154a879f941cc5c215913c P c573d0f081194f1ebaaf440a0edb5074 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c573d0f081194f1ebaaf440a0edb5074" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 42583 } } peers { permanent_uuid: "447f81cd31b74dccbf6a0d3b113a81f4" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40735 } } peers { permanent_uuid: "477e6890969f4889a60eea8e71c7201f" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 38425 } }
I20260808 18:25:16.319738 6992 raft_consensus.cc:729] T 94e20ad153154a879f941cc5c215913c P c573d0f081194f1ebaaf440a0edb5074 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c573d0f081194f1ebaaf440a0edb5074, State: Initialized, Role: FOLLOWER
I20260808 18:25:16.320274 6992 consensus_queue.cc:260] T 94e20ad153154a879f941cc5c215913c P c573d0f081194f1ebaaf440a0edb5074 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c573d0f081194f1ebaaf440a0edb5074" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 42583 } } peers { permanent_uuid: "447f81cd31b74dccbf6a0d3b113a81f4" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40735 } } peers { permanent_uuid: "477e6890969f4889a60eea8e71c7201f" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 38425 } }
I20260808 18:25:16.321179 6992 ts_tablet_manager.cc:1428] T 94e20ad153154a879f941cc5c215913c P c573d0f081194f1ebaaf440a0edb5074: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260808 18:25:16.321974 7106 heartbeater.cc:347] Connected to a master server at 127.23.72.190:37911
I20260808 18:25:16.322201 7106 heartbeater.cc:464] Registering TS with master...
I20260808 18:25:16.322654 7106 heartbeater.cc:511] Master 127.23.72.190:37911 requested a full tablet report, sending...
I20260808 18:25:16.325196 6909 ts_manager.cc:194] Registered new tserver with Master: c573d0f081194f1ebaaf440a0edb5074 (127.23.72.129:42583)
I20260808 18:25:16.329162 6909 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.129@KRBTEST.COM'} at 127.23.72.129:39379
I20260808 18:25:16.331642 7106 heartbeater.cc:503] Master 127.23.72.190:37911 was elected leader, sending a full tablet report...
Aug 08 18:25:16 dist-test-slave-5k54 krb5kdc[6294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213516, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for kudu/127.23.72.129@KRBTEST.COM
I20260808 18:25:16.340426 7114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:16.328551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55780 (local address 127.23.72.129:42583)
0808 18:25:16.328941 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:16.328952 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:16.329496 (+ 544us) server_negotiation.cc:407] Connection header received
0808 18:25:16.329553 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:16.329557 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:16.329703 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:16.329840 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:25:16.330658 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:16.331919 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:16.333071 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:16.333690 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:16.333800 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:16.336304 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:25:16.336315 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:25:16.336327 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:25:16.336354 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:25:16.338193 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:16.338673 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:16.338679 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:16.338681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:16.338775 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:16.339087 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:16.339092 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:16.339094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:16.339483 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:25:16.339701 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:16.340086 (+ 385us) server_negotiation.cc:299] Negotiation successful
0808 18:25:16.340252 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":98,"threads_started":1}
I20260808 18:25:16.340432 7113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:16.328493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:42583 (local address 127.0.0.1:55780)
0808 18:25:16.329182 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:16.329218 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:16.329380 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:16.329851 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:16.329856 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:16.330392 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:25:16.330532 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:16.330541 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:16.332039 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:16.332046 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:16.332956 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:16.332966 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:16.333814 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:16.333817 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:16.333912 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:16.334479 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:25:16.334491 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:25:16.336152 (+ 1661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:25:16.338324 (+ 2172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:16.338333 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:16.338338 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:16.338568 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:16.338874 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:16.338876 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:16.338878 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:16.338984 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:16.339672 (+ 688us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:25:16.339678 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:25:16.339851 (+ 173us) client_negotiation.cc:769] Sending connection context
0808 18:25:16.339997 (+ 146us) client_negotiation.cc:240] Negotiation successful
0808 18:25:16.340207 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":110,"threads_started":1}
I20260808 18:25:16.341467 7036 tablet_service.cc:1514] Processing DeleteTablet for tablet 94e20ad153154a879f941cc5c215913c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:25:15 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:55780
I20260808 18:25:16.342473 7115 tablet_replica.cc:330] stopping tablet replica
I20260808 18:25:16.342789 7115 raft_consensus.cc:2229] T 94e20ad153154a879f941cc5c215913c P c573d0f081194f1ebaaf440a0edb5074 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:25:16.343053 7115 raft_consensus.cc:2258] T 94e20ad153154a879f941cc5c215913c P c573d0f081194f1ebaaf440a0edb5074 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:25:16.344038 7115 ts_tablet_manager.cc:1905] T 94e20ad153154a879f941cc5c215913c P c573d0f081194f1ebaaf440a0edb5074: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:25:16.347591 7115 ts_tablet_manager.cc:1918] T 94e20ad153154a879f941cc5c215913c P c573d0f081194f1ebaaf440a0edb5074: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:25:16.347698 7115 log.cc:1196] T 94e20ad153154a879f941cc5c215913c P c573d0f081194f1ebaaf440a0edb5074: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-0/wal/wals/94e20ad153154a879f941cc5c215913c
I20260808 18:25:16.347971 7115 ts_tablet_manager.cc:1939] T 94e20ad153154a879f941cc5c215913c P c573d0f081194f1ebaaf440a0edb5074: Deleting consensus metadata
I20260808 18:25:16.349532 6893 catalog_manager.cc:4841] TS c573d0f081194f1ebaaf440a0edb5074 (127.23.72.129:42583): tablet 94e20ad153154a879f941cc5c215913c (table test-table [id=3f4ab3bf020741c5afca8cbbfa825a45]) successfully deleted
W20260808 18:25:16.431586 7111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:25:16.431850 7111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:25:16.431892 7111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:25:16.435936 7111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:25:16.436003 7111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:25:16.436100 7111 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.130
I20260808 18:25:16.441717 7111 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:38763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab
--principal=kudu/127.23.72.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.130:38425
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.72.130
--webserver_port=39023
--tserver_master_addrs=127.23.72.190:37911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.130
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:25:16.442754 7111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:25:16.443819 7111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:25:16.453224 7121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:16.453230 7123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:16.453233 7120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:25:16.453514 7111 server_base.cc:1034] running on GCE node
I20260808 18:25:16.454021 7111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:25:16.454663 7111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:25:16.455847 7111 hybrid_clock.cc:648] HybridClock initialized: now 1786213516455831 us; error 35 us; skew 500 ppm
Aug 08 18:25:16 dist-test-slave-5k54 krb5kdc[6294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213516, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:25:16.458535 7111 init.cc:377] Logged in from keytab as kudu/127.23.72.130@KRBTEST.COM (short username kudu)
I20260808 18:25:16.459486 7111 webserver.cc:460] Webserver started at http://127.23.72.130:39023/ using document root <none> and password file <none>
I20260808 18:25:16.460008 7111 fs_manager.cc:359] Metadata directory not provided
I20260808 18:25:16.460083 7111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:25:16.463897 7111 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 18:25:16.465927 7131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:16.467231 7111 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:25:16.467332 7111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-1/wal
uuid: "477e6890969f4889a60eea8e71c7201f"
format_stamp: "Formatted at 2026-08-08 18:25:15 on dist-test-slave-5k54"
server_key: "17346c4437bfba79f09bae0e10393e27"
server_key_iv: "a0b37598c0509654609f3e886bcfbe26"
server_key_version: "encryptionkey@0"
I20260808 18:25:16.467962 7111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:25:16.487339 7111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:25:16.490545 7111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:25:16.490748 7111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:25:16.491346 7111 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:25:16.492220 7111 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:25:16.492269 7111 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:16.492309 7111 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:25:16.492324 7111 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:16.501322 7111 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.130:38425
I20260808 18:25:16.501353 7254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.130:38425 every 8 connection(s)
I20260808 18:25:16.502355 7111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
I20260808 18:25:16.509099 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 7111
I20260808 18:25:16.509567 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.131:40735
--local_ip_for_outbound_sockets=127.23.72.131
--tserver_master_addrs=127.23.72.190:37911
--webserver_port=40977
--webserver_interface=127.23.72.131
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab
--principal=kudu/127.23.72.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:38763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 18:25:16 dist-test-slave-5k54 krb5kdc[6294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213516, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.130@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:25:16.516209 7109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:16.504165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:43131 (local address 127.23.72.190:37911)
0808 18:25:16.504343 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:16.504348 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:16.505176 (+ 828us) server_negotiation.cc:407] Connection header received
0808 18:25:16.506485 (+ 1309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:16.506487 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:16.506537 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:16.506614 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:25:16.507805 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:16.508337 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:16.509540 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:16.509939 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:16.510061 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:16.512478 (+ 2417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:25:16.512483 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:25:16.512486 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:25:16.512517 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:25:16.514188 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:16.514689 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:16.514694 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:16.514696 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:16.514783 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:16.515114 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:16.515117 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:16.515119 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:16.515303 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:25:16.515408 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:16.515893 (+ 485us) server_negotiation.cc:299] Negotiation successful
0808 18:25:16.516032 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 18:25:16.517086 7257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:16.504565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.130:43131)
0808 18:25:16.505022 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:16.505064 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:16.506282 (+ 1218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:16.506775 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:16.506781 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:16.507147 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:25:16.507677 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:16.507691 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:16.508465 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:16.508469 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:16.509418 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:16.509427 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:16.510085 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:16.510088 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:16.510229 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:16.510844 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:25:16.510859 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:25:16.512310 (+ 1451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:25:16.514321 (+ 2011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:16.514326 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:16.514337 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:16.514581 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:16.514873 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:16.514877 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:16.514879 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:16.515018 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:16.515405 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:25:16.515411 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:25:16.515665 (+ 254us) client_negotiation.cc:769] Sending connection context
0808 18:25:16.515850 (+ 185us) client_negotiation.cc:240] Negotiation successful
0808 18:25:16.516169 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":124,"threads_started":1}
I20260808 18:25:16.518260 7255 heartbeater.cc:347] Connected to a master server at 127.23.72.190:37911
I20260808 18:25:16.518534 7255 heartbeater.cc:464] Registering TS with master...
I20260808 18:25:16.519176 7255 heartbeater.cc:511] Master 127.23.72.190:37911 requested a full tablet report, sending...
I20260808 18:25:16.520390 6909 ts_manager.cc:194] Registered new tserver with Master: 477e6890969f4889a60eea8e71c7201f (127.23.72.130:38425)
I20260808 18:25:16.520951 6909 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.130@KRBTEST.COM'} at 127.23.72.130:43131
W20260808 18:25:16.624483 7258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:25:16.624737 7258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:25:16.624804 7258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:25:16.628901 7258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:25:16.628959 7258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:25:16.629019 7258 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.131
I20260808 18:25:16.634150 7258 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:38763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab
--principal=kudu/127.23.72.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.131:40735
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.72.131
--webserver_port=40977
--tserver_master_addrs=127.23.72.190:37911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7258
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.131
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:25:16.635200 7258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:25:16.636250 7258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:25:16.646008 7267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:16.646021 7265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:16.646037 7264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:25:16.646262 7258 server_base.cc:1034] running on GCE node
I20260808 18:25:16.646781 7258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:25:16.647403 7258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:25:16.648610 7258 hybrid_clock.cc:648] HybridClock initialized: now 1786213516648594 us; error 28 us; skew 500 ppm
Aug 08 18:25:16 dist-test-slave-5k54 krb5kdc[6294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213516, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:25:16.651466 7258 init.cc:377] Logged in from keytab as kudu/127.23.72.131@KRBTEST.COM (short username kudu)
I20260808 18:25:16.652319 7258 webserver.cc:460] Webserver started at http://127.23.72.131:40977/ using document root <none> and password file <none>
I20260808 18:25:16.652859 7258 fs_manager.cc:359] Metadata directory not provided
I20260808 18:25:16.652935 7258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:25:16.656889 7258 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:25:16.658887 7275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:16.660218 7258 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:25:16.660316 7258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-2/wal
uuid: "447f81cd31b74dccbf6a0d3b113a81f4"
format_stamp: "Formatted at 2026-08-08 18:25:15 on dist-test-slave-5k54"
server_key: "a2082b064639115d217cba86fcedb5ea"
server_key_iv: "b6a58bf6c75d31a6cf218a7c810d308a"
server_key_version: "encryptionkey@0"
I20260808 18:25:16.660876 7258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:25:16.686071 7258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:25:16.689581 7258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:25:16.689793 7258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:25:16.690452 7258 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:25:16.691282 7258 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:25:16.691344 7258 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:16.691459 7258 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:25:16.691514 7258 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:16.700814 7258 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.131:40735
I20260808 18:25:16.700845 7396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.131:40735 every 8 connection(s)
I20260808 18:25:16.701841 7258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
I20260808 18:25:16.707751 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 7258
Aug 08 18:25:16 dist-test-slave-5k54 krb5kdc[6294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213516, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.131@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:25:16.717123 7109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:16.703727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:44439 (local address 127.23.72.190:37911)
0808 18:25:16.703874 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:16.703878 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:16.704717 (+ 839us) server_negotiation.cc:407] Connection header received
0808 18:25:16.706132 (+ 1415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:16.706135 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:16.706194 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:16.706297 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:25:16.707487 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:16.708093 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:16.709731 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:16.710032 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:16.710129 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:16.713101 (+ 2972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:25:16.713104 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:25:16.713106 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:25:16.713134 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:25:16.714892 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:16.715365 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:16.715369 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:16.715370 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:16.715488 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:16.715885 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:16.715888 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:16.715890 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:16.716129 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:25:16.716200 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:16.716800 (+ 600us) server_negotiation.cc:299] Negotiation successful
0808 18:25:16.716934 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52,"spinlock_wait_cycles":3840}
I20260808 18:25:16.717124 7401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:16.708450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36532 (local address 127.23.72.190:37911)
0808 18:25:16.708714 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:16.708717 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:16.708903 (+ 186us) server_negotiation.cc:407] Connection header received
0808 18:25:16.709024 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:16.709026 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:16.709066 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:16.709137 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:25:16.710032 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:16.710620 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:16.711920 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:16.712290 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:16.712413 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:16.713717 (+ 1304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:25:16.713723 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:25:16.713726 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:25:16.713767 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:25:16.715390 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:16.715918 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:16.715922 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:16.715927 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:16.715993 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:16.716287 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:16.716291 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:16.716292 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:16.716438 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:25:16.716590 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:16.716843 (+ 253us) server_negotiation.cc:299] Negotiation successful
0808 18:25:16.716991 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":193,"spinlock_wait_cycles":1408,"thread_start_us":95,"threads_started":1}
I20260808 18:25:16.718130 7399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:16.704125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.131:44439)
0808 18:25:16.704566 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:16.704607 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:16.705920 (+ 1313us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:16.706481 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:16.706488 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:16.706845 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:25:16.707313 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:16.707335 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:16.708214 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:16.708218 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:16.709578 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:16.709592 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:16.710185 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:16.710188 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:16.710339 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:16.710968 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:25:16.710988 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:25:16.712913 (+ 1925us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:25:16.715003 (+ 2090us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:16.715007 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:16.715019 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:16.715267 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:16.715579 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:16.715585 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:16.715589 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:16.715762 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:16.716229 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:25:16.716239 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:25:16.716522 (+ 283us) client_negotiation.cc:769] Sending connection context
0808 18:25:16.716738 (+ 216us) client_negotiation.cc:240] Negotiation successful
0808 18:25:16.717073 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":120,"threads_started":1}
I20260808 18:25:16.719274 7397 heartbeater.cc:347] Connected to a master server at 127.23.72.190:37911
I20260808 18:25:16.719563 7397 heartbeater.cc:464] Registering TS with master...
I20260808 18:25:16.720044 7397 heartbeater.cc:511] Master 127.23.72.190:37911 requested a full tablet report, sending...
I20260808 18:25:16.721029 6908 ts_manager.cc:194] Registered new tserver with Master: 447f81cd31b74dccbf6a0d3b113a81f4 (127.23.72.131:40735)
I20260808 18:25:16.721588 6908 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.131@KRBTEST.COM'} at 127.23.72.131:44439
I20260808 18:25:16.730754 23842 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:25:16.735209 7401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:16.731418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36534 (local address 127.23.72.190:37911)
0808 18:25:16.731563 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:16.731568 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:16.731766 (+ 198us) server_negotiation.cc:407] Connection header received
0808 18:25:16.731910 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:16.731912 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:16.732006 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:16.732091 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:25:16.732286 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:16.732796 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:16.733686 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:16.734002 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:16.734098 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:16.734375 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:25:16.734798 (+ 423us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:25:16.734904 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:16.734955 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 18:25:16.735012 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 18:25:16.735842 6908 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:36534:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:25:16.739042 6908 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:25:16.746723 7036 tablet_service.cc:1467] Processing CreateTablet for tablet b1581f1b34b4461e9aa174a0c0bd5b44 (DEFAULT_TABLE table=test-table [id=ec0260d09f194ad7a883046e2d63e4fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:25:16.747480 7036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1581f1b34b4461e9aa174a0c0bd5b44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:25:16.750306 6992 tablet_bootstrap.cc:492] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074: Bootstrap starting.
I20260808 18:25:16.751194 6992 tablet_bootstrap.cc:654] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074: Neither blocks nor log segments found. Creating new log.
I20260808 18:25:16.751658 7406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:16.745898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56128 (local address 127.23.72.131:40735)
0808 18:25:16.746464 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:16.746476 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:16.746499 (+ 23us) server_negotiation.cc:407] Connection header received
0808 18:25:16.746575 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:16.746580 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:16.746796 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:16.746981 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:25:16.747234 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:16.748396 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:16.749890 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:16.751010 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:16.751122 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:16.751269 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:16.751381 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 18:25:16.751500 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":179,"threads_started":1}
I20260808 18:25:16.751677 7407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:16.746124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.130:38425 (local address 127.0.0.1:34668)
0808 18:25:16.746362 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:16.746377 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:16.746463 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:16.747356 (+ 893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:16.747359 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:16.747377 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:25:16.747488 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:16.747496 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:16.749126 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:16.749131 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:16.749912 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:16.749922 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:16.751173 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:16.751178 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:16.751274 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:16.751304 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 18:25:16.751367 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 18:25:16.751469 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":111,"threads_started":1}
I20260808 18:25:16.751701 7113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:16.745693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.131:40735 (local address 127.0.0.1:56128)
0808 18:25:16.745904 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:16.745920 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:16.746002 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:16.746975 (+ 973us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:16.746979 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:16.746998 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:25:16.747095 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:16.747100 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:16.748541 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:16.748545 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:16.749755 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:16.749766 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:16.751131 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:16.751134 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:16.751224 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:16.751267 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 18:25:16.751353 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 18:25:16.751459 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"mutex_wait_us":1}
I20260808 18:25:16.752377 7408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:16.746563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34668 (local address 127.23.72.130:38425)
0808 18:25:16.746882 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:16.746892 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:16.746929 (+ 37us) server_negotiation.cc:407] Connection header received
0808 18:25:16.747004 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:16.747009 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:16.747207 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:16.747370 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:25:16.748087 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:16.748999 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:16.750046 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:16.751021 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:16.751902 (+ 881us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:16.752055 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:16.752111 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 18:25:16.752204 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":130,"threads_started":1}
I20260808 18:25:16.752840 6992 tablet_bootstrap.cc:492] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074: No bootstrap required, opened a new log
I20260808 18:25:16.752935 6992 ts_tablet_manager.cc:1397] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260808 18:25:16.753394 6992 raft_consensus.cc:348] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "447f81cd31b74dccbf6a0d3b113a81f4" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40735 } } peers { permanent_uuid: "477e6890969f4889a60eea8e71c7201f" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 38425 } } peers { permanent_uuid: "c573d0f081194f1ebaaf440a0edb5074" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 42583 } }
I20260808 18:25:16.753505 6992 raft_consensus.cc:374] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:25:16.753543 6992 raft_consensus.cc:729] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c573d0f081194f1ebaaf440a0edb5074, State: Initialized, Role: FOLLOWER
I20260808 18:25:16.753671 6992 consensus_queue.cc:260] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "447f81cd31b74dccbf6a0d3b113a81f4" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40735 } } peers { permanent_uuid: "477e6890969f4889a60eea8e71c7201f" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 38425 } } peers { permanent_uuid: "c573d0f081194f1ebaaf440a0edb5074" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 42583 } }
I20260808 18:25:16.753934 6992 ts_tablet_manager.cc:1428] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 18:25:16.755046 7325 tablet_service.cc:1467] Processing CreateTablet for tablet b1581f1b34b4461e9aa174a0c0bd5b44 (DEFAULT_TABLE table=test-table [id=ec0260d09f194ad7a883046e2d63e4fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:25:16.755342 7185 tablet_service.cc:1467] Processing CreateTablet for tablet b1581f1b34b4461e9aa174a0c0bd5b44 (DEFAULT_TABLE table=test-table [id=ec0260d09f194ad7a883046e2d63e4fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:25:16.756158 7325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1581f1b34b4461e9aa174a0c0bd5b44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:25:16.756362 7185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1581f1b34b4461e9aa174a0c0bd5b44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:25:16.762714 7410 tablet_bootstrap.cc:492] T b1581f1b34b4461e9aa174a0c0bd5b44 P 447f81cd31b74dccbf6a0d3b113a81f4: Bootstrap starting.
I20260808 18:25:16.762809 7411 tablet_bootstrap.cc:492] T b1581f1b34b4461e9aa174a0c0bd5b44 P 477e6890969f4889a60eea8e71c7201f: Bootstrap starting.
I20260808 18:25:16.765131 7410 tablet_bootstrap.cc:654] T b1581f1b34b4461e9aa174a0c0bd5b44 P 447f81cd31b74dccbf6a0d3b113a81f4: Neither blocks nor log segments found. Creating new log.
I20260808 18:25:16.765614 7411 tablet_bootstrap.cc:654] T b1581f1b34b4461e9aa174a0c0bd5b44 P 477e6890969f4889a60eea8e71c7201f: Neither blocks nor log segments found. Creating new log.
I20260808 18:25:16.766299 7410 log.cc:824] T b1581f1b34b4461e9aa174a0c0bd5b44 P 447f81cd31b74dccbf6a0d3b113a81f4: Log is configured to *not* fsync() on all Append() calls
I20260808 18:25:16.766299 7411 log.cc:824] T b1581f1b34b4461e9aa174a0c0bd5b44 P 477e6890969f4889a60eea8e71c7201f: Log is configured to *not* fsync() on all Append() calls
I20260808 18:25:16.768036 7411 tablet_bootstrap.cc:492] T b1581f1b34b4461e9aa174a0c0bd5b44 P 477e6890969f4889a60eea8e71c7201f: No bootstrap required, opened a new log
I20260808 18:25:16.768057 7410 tablet_bootstrap.cc:492] T b1581f1b34b4461e9aa174a0c0bd5b44 P 447f81cd31b74dccbf6a0d3b113a81f4: No bootstrap required, opened a new log
I20260808 18:25:16.768242 7410 ts_tablet_manager.cc:1397] T b1581f1b34b4461e9aa174a0c0bd5b44 P 447f81cd31b74dccbf6a0d3b113a81f4: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 18:25:16.768244 7411 ts_tablet_manager.cc:1397] T b1581f1b34b4461e9aa174a0c0bd5b44 P 477e6890969f4889a60eea8e71c7201f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 18:25:16.771129 7411 raft_consensus.cc:348] T b1581f1b34b4461e9aa174a0c0bd5b44 P 477e6890969f4889a60eea8e71c7201f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "447f81cd31b74dccbf6a0d3b113a81f4" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40735 } } peers { permanent_uuid: "477e6890969f4889a60eea8e71c7201f" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 38425 } } peers { permanent_uuid: "c573d0f081194f1ebaaf440a0edb5074" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 42583 } }
I20260808 18:25:16.771409 7411 raft_consensus.cc:374] T b1581f1b34b4461e9aa174a0c0bd5b44 P 477e6890969f4889a60eea8e71c7201f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:25:16.771512 7411 raft_consensus.cc:729] T b1581f1b34b4461e9aa174a0c0bd5b44 P 477e6890969f4889a60eea8e71c7201f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 477e6890969f4889a60eea8e71c7201f, State: Initialized, Role: FOLLOWER
I20260808 18:25:16.771314 7410 raft_consensus.cc:348] T b1581f1b34b4461e9aa174a0c0bd5b44 P 447f81cd31b74dccbf6a0d3b113a81f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "447f81cd31b74dccbf6a0d3b113a81f4" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40735 } } peers { permanent_uuid: "477e6890969f4889a60eea8e71c7201f" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 38425 } } peers { permanent_uuid: "c573d0f081194f1ebaaf440a0edb5074" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 42583 } }
I20260808 18:25:16.771586 7410 raft_consensus.cc:374] T b1581f1b34b4461e9aa174a0c0bd5b44 P 447f81cd31b74dccbf6a0d3b113a81f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:25:16.771649 7410 raft_consensus.cc:729] T b1581f1b34b4461e9aa174a0c0bd5b44 P 447f81cd31b74dccbf6a0d3b113a81f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 447f81cd31b74dccbf6a0d3b113a81f4, State: Initialized, Role: FOLLOWER
I20260808 18:25:16.771960 7411 consensus_queue.cc:260] T b1581f1b34b4461e9aa174a0c0bd5b44 P 477e6890969f4889a60eea8e71c7201f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "447f81cd31b74dccbf6a0d3b113a81f4" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40735 } } peers { permanent_uuid: "477e6890969f4889a60eea8e71c7201f" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 38425 } } peers { permanent_uuid: "c573d0f081194f1ebaaf440a0edb5074" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 42583 } }
I20260808 18:25:16.772074 7410 consensus_queue.cc:260] T b1581f1b34b4461e9aa174a0c0bd5b44 P 447f81cd31b74dccbf6a0d3b113a81f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "447f81cd31b74dccbf6a0d3b113a81f4" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40735 } } peers { permanent_uuid: "477e6890969f4889a60eea8e71c7201f" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 38425 } } peers { permanent_uuid: "c573d0f081194f1ebaaf440a0edb5074" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 42583 } }
I20260808 18:25:16.772706 7397 heartbeater.cc:503] Master 127.23.72.190:37911 was elected leader, sending a full tablet report...
I20260808 18:25:16.772886 7410 ts_tablet_manager.cc:1428] T b1581f1b34b4461e9aa174a0c0bd5b44 P 447f81cd31b74dccbf6a0d3b113a81f4: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 18:25:16.772897 7411 ts_tablet_manager.cc:1428] T b1581f1b34b4461e9aa174a0c0bd5b44 P 477e6890969f4889a60eea8e71c7201f: Time spent starting tablet: real 0.005s user 0.002s sys 0.003s
I20260808 18:25:16.773224 7255 heartbeater.cc:503] Master 127.23.72.190:37911 was elected leader, sending a full tablet report...
W20260808 18:25:16.800902 7107 tablet.cc:2369] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 18:25:16.952836 7398 tablet.cc:2369] T b1581f1b34b4461e9aa174a0c0bd5b44 P 447f81cd31b74dccbf6a0d3b113a81f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 18:25:17.003603 7256 tablet.cc:2369] T b1581f1b34b4461e9aa174a0c0bd5b44 P 477e6890969f4889a60eea8e71c7201f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:25:17.131973 7112 raft_consensus.cc:482] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:25:17.132228 7112 raft_consensus.cc:504] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "447f81cd31b74dccbf6a0d3b113a81f4" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40735 } } peers { permanent_uuid: "477e6890969f4889a60eea8e71c7201f" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 38425 } } peers { permanent_uuid: "c573d0f081194f1ebaaf440a0edb5074" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 42583 } }
I20260808 18:25:17.133483 7112 leader_election.cc:288] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 447f81cd31b74dccbf6a0d3b113a81f4 (127.23.72.131:40735), 477e6890969f4889a60eea8e71c7201f (127.23.72.130:38425)
I20260808 18:25:17.137496 7406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:17.133829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:38837 (local address 127.23.72.131:40735)
0808 18:25:17.133968 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:17.133973 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:17.134077 (+ 104us) server_negotiation.cc:407] Connection header received
0808 18:25:17.134372 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:17.134374 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:17.134419 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:17.134509 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:25:17.134822 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:17.135298 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:17.136389 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:17.136845 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:17.136943 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:17.137018 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:17.137265 (+ 247us) server_negotiation.cc:299] Negotiation successful
0808 18:25:17.137349 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 18:25:17.137504 7417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:17.133828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.130:38425 (local address 127.23.72.129:42443)
0808 18:25:17.134101 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:17.134116 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:17.134220 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:17.134505 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:17.134508 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:17.134524 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:25:17.134669 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:17.134675 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:17.135596 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:17.135599 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:17.136359 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:17.136371 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:17.137049 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:17.137053 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:17.137125 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:17.137195 (+ 70us) client_negotiation.cc:769] Sending connection context
0808 18:25:17.137288 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 18:25:17.137356 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"mutex_wait_us":45,"spinlock_wait_cycles":16000,"thread_start_us":79,"threads_started":1}
I20260808 18:25:17.137523 7408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:17.134097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:42443 (local address 127.23.72.130:38425)
0808 18:25:17.134253 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:17.134268 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:17.134286 (+ 18us) server_negotiation.cc:407] Connection header received
0808 18:25:17.134348 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:17.134351 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:17.134402 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:17.134463 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:25:17.134790 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:17.135329 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:17.136496 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:17.136890 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:17.137046 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:17.137097 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:17.137320 (+ 223us) server_negotiation.cc:299] Negotiation successful
0808 18:25:17.137421 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 18:25:17.137513 7416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:17.133674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.131:40735 (local address 127.23.72.129:38837)
0808 18:25:17.133976 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:17.133993 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:17.134208 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:17.134542 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:17.134545 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:17.134586 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:25:17.134686 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:17.134691 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:17.135413 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:17.135417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:17.136265 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:17.136275 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:17.136992 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:17.136999 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:17.137087 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:17.137168 (+ 81us) client_negotiation.cc:769] Sending connection context
0808 18:25:17.137229 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 18:25:17.137329 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":7,"spinlock_wait_cycles":896,"thread_start_us":97,"threads_started":1}
I20260808 18:25:17.138300 7205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1581f1b34b4461e9aa174a0c0bd5b44" candidate_uuid: "c573d0f081194f1ebaaf440a0edb5074" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "477e6890969f4889a60eea8e71c7201f" is_pre_election: true
I20260808 18:25:17.138386 7346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1581f1b34b4461e9aa174a0c0bd5b44" candidate_uuid: "c573d0f081194f1ebaaf440a0edb5074" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "447f81cd31b74dccbf6a0d3b113a81f4" is_pre_election: true
I20260808 18:25:17.138636 7205 raft_consensus.cc:2454] T b1581f1b34b4461e9aa174a0c0bd5b44 P 477e6890969f4889a60eea8e71c7201f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c573d0f081194f1ebaaf440a0edb5074 in term 0.
I20260808 18:25:17.138692 7346 raft_consensus.cc:2454] T b1581f1b34b4461e9aa174a0c0bd5b44 P 447f81cd31b74dccbf6a0d3b113a81f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c573d0f081194f1ebaaf440a0edb5074 in term 0.
I20260808 18:25:17.139091 6989 leader_election.cc:302] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 477e6890969f4889a60eea8e71c7201f, c573d0f081194f1ebaaf440a0edb5074; no voters:
I20260808 18:25:17.139400 7112 raft_consensus.cc:2781] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:25:17.139573 7112 raft_consensus.cc:482] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:25:17.139631 7112 raft_consensus.cc:3037] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:25:17.140825 7112 raft_consensus.cc:504] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "447f81cd31b74dccbf6a0d3b113a81f4" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40735 } } peers { permanent_uuid: "477e6890969f4889a60eea8e71c7201f" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 38425 } } peers { permanent_uuid: "c573d0f081194f1ebaaf440a0edb5074" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 42583 } }
I20260808 18:25:17.141218 7112 leader_election.cc:288] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [CANDIDATE]: Term 1 election: Requested vote from peers 447f81cd31b74dccbf6a0d3b113a81f4 (127.23.72.131:40735), 477e6890969f4889a60eea8e71c7201f (127.23.72.130:38425)
I20260808 18:25:17.141616 7346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1581f1b34b4461e9aa174a0c0bd5b44" candidate_uuid: "c573d0f081194f1ebaaf440a0edb5074" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "447f81cd31b74dccbf6a0d3b113a81f4"
I20260808 18:25:17.141626 7205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1581f1b34b4461e9aa174a0c0bd5b44" candidate_uuid: "c573d0f081194f1ebaaf440a0edb5074" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "477e6890969f4889a60eea8e71c7201f"
I20260808 18:25:17.141726 7346 raft_consensus.cc:3037] T b1581f1b34b4461e9aa174a0c0bd5b44 P 447f81cd31b74dccbf6a0d3b113a81f4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:25:17.141767 7205 raft_consensus.cc:3037] T b1581f1b34b4461e9aa174a0c0bd5b44 P 477e6890969f4889a60eea8e71c7201f [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:25:17.142774 7346 raft_consensus.cc:2454] T b1581f1b34b4461e9aa174a0c0bd5b44 P 447f81cd31b74dccbf6a0d3b113a81f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c573d0f081194f1ebaaf440a0edb5074 in term 1.
I20260808 18:25:17.142892 7205 raft_consensus.cc:2454] T b1581f1b34b4461e9aa174a0c0bd5b44 P 477e6890969f4889a60eea8e71c7201f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c573d0f081194f1ebaaf440a0edb5074 in term 1.
I20260808 18:25:17.143090 6988 leader_election.cc:302] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 447f81cd31b74dccbf6a0d3b113a81f4, c573d0f081194f1ebaaf440a0edb5074; no voters:
I20260808 18:25:17.143267 7112 raft_consensus.cc:2781] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:25:17.143462 7112 raft_consensus.cc:686] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [term 1 LEADER]: Becoming Leader. State: Replica: c573d0f081194f1ebaaf440a0edb5074, State: Running, Role: LEADER
I20260808 18:25:17.143719 7112 consensus_queue.cc:237] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "447f81cd31b74dccbf6a0d3b113a81f4" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40735 } } peers { permanent_uuid: "477e6890969f4889a60eea8e71c7201f" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 38425 } } peers { permanent_uuid: "c573d0f081194f1ebaaf440a0edb5074" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 42583 } }
I20260808 18:25:17.146572 6907 catalog_manager.cc:5495] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 reported cstate change: term changed from 0 to 1, leader changed from <none> to c573d0f081194f1ebaaf440a0edb5074 (127.23.72.129). New cstate: current_term: 1 leader_uuid: "c573d0f081194f1ebaaf440a0edb5074" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "447f81cd31b74dccbf6a0d3b113a81f4" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40735 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "477e6890969f4889a60eea8e71c7201f" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 38425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c573d0f081194f1ebaaf440a0edb5074" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 42583 } health_report { overall_health: HEALTHY } } }
I20260808 18:25:17.209398 7421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:17.205833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55796 (local address 127.23.72.129:42583)
0808 18:25:17.206142 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:17.206149 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:17.206161 (+ 12us) server_negotiation.cc:407] Connection header received
0808 18:25:17.206238 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:17.206240 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:17.206296 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:17.206415 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:25:17.206626 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:17.207171 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:17.207993 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:17.208225 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:17.208303 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:17.208558 (+ 255us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:25:17.209047 (+ 489us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:25:17.209145 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:17.209203 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 18:25:17.209250 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":133,"threads_started":1}
I20260808 18:25:17.213349 7346 raft_consensus.cc:1261] T b1581f1b34b4461e9aa174a0c0bd5b44 P 447f81cd31b74dccbf6a0d3b113a81f4 [term 1 FOLLOWER]: Refusing update from remote peer c573d0f081194f1ebaaf440a0edb5074: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:25:17.213343 7205 raft_consensus.cc:1261] T b1581f1b34b4461e9aa174a0c0bd5b44 P 477e6890969f4889a60eea8e71c7201f [term 1 FOLLOWER]: Refusing update from remote peer c573d0f081194f1ebaaf440a0edb5074: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:25:17.214098 7418 consensus_queue.cc:1035] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [LEADER]: Connected to new peer: Peer: permanent_uuid: "447f81cd31b74dccbf6a0d3b113a81f4" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40735 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:25:17.214288 7112 consensus_queue.cc:1035] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [LEADER]: Connected to new peer: Peer: permanent_uuid: "477e6890969f4889a60eea8e71c7201f" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 38425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:25:17.220356 7422 mvcc.cc:205] Tried to move back new op lower bound from 7316330566499532800 to 7316330566223192064. Current Snapshot: MvccSnapshot[applied={T|T < 7316330566499532800}]
I20260808 18:25:17.221545 7424 mvcc.cc:205] Tried to move back new op lower bound from 7316330566499532800 to 7316330566223192064. Current Snapshot: MvccSnapshot[applied={T|T < 7316330566499532800}]
I20260808 18:25:17.229125 6907 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:36534:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:25:17.229266 6907 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:36534:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:25:17.231546 6907 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3: Sending DeleteTablet for 3 replicas of tablet b1581f1b34b4461e9aa174a0c0bd5b44
I20260808 18:25:17.232267 7325 tablet_service.cc:1514] Processing DeleteTablet for tablet b1581f1b34b4461e9aa174a0c0bd5b44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:25:17 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:56128
I20260808 18:25:17.232296 7185 tablet_service.cc:1514] Processing DeleteTablet for tablet b1581f1b34b4461e9aa174a0c0bd5b44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:25:17 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:34668
I20260808 18:25:17.232501 7036 tablet_service.cc:1514] Processing DeleteTablet for tablet b1581f1b34b4461e9aa174a0c0bd5b44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:25:17 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:55780
I20260808 18:25:17.232748 7433 tablet_replica.cc:330] T b1581f1b34b4461e9aa174a0c0bd5b44 P 447f81cd31b74dccbf6a0d3b113a81f4: stopping tablet replica
I20260808 18:25:17.232812 7435 tablet_replica.cc:330] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074: stopping tablet replica
I20260808 18:25:17.232930 7435 raft_consensus.cc:2229] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:25:17.233036 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 6950
I20260808 18:25:17.233063 7433 raft_consensus.cc:2229] T b1581f1b34b4461e9aa174a0c0bd5b44 P 447f81cd31b74dccbf6a0d3b113a81f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:25:17.233361 7433 raft_consensus.cc:2258] T b1581f1b34b4461e9aa174a0c0bd5b44 P 447f81cd31b74dccbf6a0d3b113a81f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:25:17.235143 7433 ts_tablet_manager.cc:1905] T b1581f1b34b4461e9aa174a0c0bd5b44 P 447f81cd31b74dccbf6a0d3b113a81f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:25:17.237406 7433 ts_tablet_manager.cc:1918] T b1581f1b34b4461e9aa174a0c0bd5b44 P 447f81cd31b74dccbf6a0d3b113a81f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:25:17.237524 7433 log.cc:1196] T b1581f1b34b4461e9aa174a0c0bd5b44 P 447f81cd31b74dccbf6a0d3b113a81f4: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-2/wal/wals/b1581f1b34b4461e9aa174a0c0bd5b44
I20260808 18:25:17.237864 7433 ts_tablet_manager.cc:1939] T b1581f1b34b4461e9aa174a0c0bd5b44 P 447f81cd31b74dccbf6a0d3b113a81f4: Deleting consensus metadata
I20260808 18:25:17.239233 6894 catalog_manager.cc:4841] TS 447f81cd31b74dccbf6a0d3b113a81f4 (127.23.72.131:40735): tablet b1581f1b34b4461e9aa174a0c0bd5b44 (table test-table [id=ec0260d09f194ad7a883046e2d63e4fc]) successfully deleted
I20260808 18:25:17.239646 7434 tablet_replica.cc:330] T b1581f1b34b4461e9aa174a0c0bd5b44 P 477e6890969f4889a60eea8e71c7201f: stopping tablet replica
I20260808 18:25:17.239974 7434 raft_consensus.cc:2229] T b1581f1b34b4461e9aa174a0c0bd5b44 P 477e6890969f4889a60eea8e71c7201f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:25:17.240279 7434 raft_consensus.cc:2258] T b1581f1b34b4461e9aa174a0c0bd5b44 P 477e6890969f4889a60eea8e71c7201f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:25:17.241565 7434 ts_tablet_manager.cc:1905] T b1581f1b34b4461e9aa174a0c0bd5b44 P 477e6890969f4889a60eea8e71c7201f: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 18:25:17.243854 6893 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.72.129:42583 (error 108)
I20260808 18:25:17.243989 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 7111
I20260808 18:25:17.248291 7113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:17.248002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:42583 (local address 127.0.0.1:55812)
0808 18:25:17.248140 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:17.248210 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.129:42583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 18:25:17.248566 6893 catalog_manager.cc:4584] TS c573d0f081194f1ebaaf440a0edb5074 (127.23.72.129:42583): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b1581f1b34b4461e9aa174a0c0bd5b44: Network error: Client connection negotiation failed: client connection to 127.23.72.129:42583: connect: Connection refused (error 111)
I20260808 18:25:17.252063 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 7258
I20260808 18:25:17.252369 7113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:17.252108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.130:38425 (local address 127.0.0.1:34684)
0808 18:25:17.252221 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:17.252278 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.130:38425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 18:25:17.261354 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 6874
I20260808 18:25:17.269647 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:37911
--webserver_interface=127.23.72.190
--webserver_port=46233
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:38763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:37911
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:25:17.394410 7437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:25:17.394691 7437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:25:17.394737 7437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:25:17.399142 7437 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:25:17.399206 7437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:25:17.399227 7437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:25:17.399246 7437 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:25:17.399263 7437 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:25:17.405174 7437 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:38763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:37911
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:37911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.72.190
--webserver_port=46233
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7437
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:25:17.406284 7437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:25:17.407410 7437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:25:17.416639 7442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:17.416600 7445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:25:17.417027 7437 server_base.cc:1034] running on GCE node
W20260808 18:25:17.416596 7443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:25:17.417879 7437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:25:17.419057 7437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:25:17.420245 7437 hybrid_clock.cc:648] HybridClock initialized: now 1786213517420225 us; error 31 us; skew 500 ppm
Aug 08 18:25:17 dist-test-slave-5k54 krb5kdc[6294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213517, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:25:17.423192 7437 init.cc:377] Logged in from keytab as kudu/127.23.72.190@KRBTEST.COM (short username kudu)
I20260808 18:25:17.424196 7437 webserver.cc:460] Webserver started at http://127.23.72.190:46233/ using document root <none> and password file <none>
I20260808 18:25:17.424731 7437 fs_manager.cc:359] Metadata directory not provided
I20260808 18:25:17.424803 7437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:25:17.428761 7437 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:25:17.431033 7452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:17.432400 7437 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:25:17.432508 7437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "3c4a0053fcaf4116ba005f16ef7094d3"
format_stamp: "Formatted at 2026-08-08 18:25:14 on dist-test-slave-5k54"
server_key: "aeebdb9a77fb4d8b58197855efd3d6d2"
server_key_iv: "53acdc05aa4f45c29539c240224385c6"
server_key_version: "encryptionkey@0"
I20260808 18:25:17.433112 7437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:25:17.447379 7437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:25:17.450407 7437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:25:17.450564 7437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:25:17.457971 7437 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.190:37911
I20260808 18:25:17.458307 7506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.190:37911 every 8 connection(s)
I20260808 18:25:17.459146 7437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:25:17.463539 7507 sys_catalog.cc:263] Verifying existing consensus state
I20260808 18:25:17.465137 7507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3: Bootstrap starting.
I20260808 18:25:17.467317 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 7437
I20260808 18:25:17.467792 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.129:42583
--local_ip_for_outbound_sockets=127.23.72.129
--tserver_master_addrs=127.23.72.190:37911
--webserver_port=42033
--webserver_interface=127.23.72.129
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab
--principal=kudu/127.23.72.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:38763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:25:17.483068 7507 log.cc:824] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3: Log is configured to *not* fsync() on all Append() calls
I20260808 18:25:17.495036 7507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 18:25:17.495474 7507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3: Bootstrap complete.
I20260808 18:25:17.499543 7507 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c4a0053fcaf4116ba005f16ef7094d3" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 37911 } }
I20260808 18:25:17.499950 7507 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c4a0053fcaf4116ba005f16ef7094d3, State: Initialized, Role: FOLLOWER
I20260808 18:25:17.500480 7507 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c4a0053fcaf4116ba005f16ef7094d3" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 37911 } }
I20260808 18:25:17.500625 7507 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:25:17.500722 7507 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:25:17.500823 7507 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [term 2 FOLLOWER]: Advancing to term 3
I20260808 18:25:17.502746 7507 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c4a0053fcaf4116ba005f16ef7094d3" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 37911 } }
I20260808 18:25:17.503082 7507 leader_election.cc:302] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c4a0053fcaf4116ba005f16ef7094d3; no voters:
I20260808 18:25:17.503418 7507 leader_election.cc:288] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 18:25:17.503526 7511 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 18:25:17.503746 7511 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [term 3 LEADER]: Becoming Leader. State: Replica: 3c4a0053fcaf4116ba005f16ef7094d3, State: Running, Role: LEADER
I20260808 18:25:17.504115 7511 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c4a0053fcaf4116ba005f16ef7094d3" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 37911 } }
I20260808 18:25:17.504733 7507 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:25:17.505518 7513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c4a0053fcaf4116ba005f16ef7094d3. Latest consensus state: current_term: 3 leader_uuid: "3c4a0053fcaf4116ba005f16ef7094d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c4a0053fcaf4116ba005f16ef7094d3" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 37911 } } }
I20260808 18:25:17.505939 7513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [sys.catalog]: This master's current role is: LEADER
I20260808 18:25:17.505827 7512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "3c4a0053fcaf4116ba005f16ef7094d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c4a0053fcaf4116ba005f16ef7094d3" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 37911 } } }
I20260808 18:25:17.506283 7512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c4a0053fcaf4116ba005f16ef7094d3 [sys.catalog]: This master's current role is: LEADER
I20260808 18:25:17.506623 7522 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:25:17.511739 7522 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:25:17.512419 7522 catalog_manager.cc:1259] Loaded cluster ID: a4d84cc9953c490ea6a9eeee5225bbea
I20260808 18:25:17.512480 7522 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260808 18:25:17.513579 7522 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f8a27d8c9ad google::LogMessage::Fail() at ??:0
@ 0x7f8a27d8f88a google::LogMessage::SendToLog() at ??:0
@ 0x7f8a27d8c4de google::LogMessage::Flush() at ??:0
@ 0x7f8a27d8da19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f8a326cbfdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f8a326ccd2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f8a326c9579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f8a3270d640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f8a29fde3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f8a29fdec83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f8a29fe03f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f8a29fcf512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f8a2b6136db start_thread at ??:0
@ 0x7f8a2688a61f clone at ??:0
W20260808 18:25:17.609741 7509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:25:17.609992 7509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:25:17.610037 7509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:25:17.614077 7509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:25:17.614133 7509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:25:17.614213 7509 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.129
I20260808 18:25:17.619983 7509 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:38763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab
--principal=kudu/127.23.72.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.129:42583
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.72.129
--webserver_port=42033
--tserver_master_addrs=127.23.72.190:37911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.129
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:25:17.621049 7509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:25:17.622149 7509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:25:17.631816 7550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:17.631844 7551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:17.631906 7553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:25:17.632253 7509 server_base.cc:1034] running on GCE node
I20260808 18:25:17.632665 7509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:25:17.633351 7509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:25:17.634526 7509 hybrid_clock.cc:648] HybridClock initialized: now 1786213517634500 us; error 46 us; skew 500 ppm
Aug 08 18:25:17 dist-test-slave-5k54 krb5kdc[6294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213517, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:25:17.637504 7509 init.cc:377] Logged in from keytab as kudu/127.23.72.129@KRBTEST.COM (short username kudu)
I20260808 18:25:17.638372 7509 webserver.cc:460] Webserver started at http://127.23.72.129:42033/ using document root <none> and password file <none>
I20260808 18:25:17.638942 7509 fs_manager.cc:359] Metadata directory not provided
I20260808 18:25:17.639031 7509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:25:17.643031 7509 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 18:25:17.645385 7560 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:17.646709 7509 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 18:25:17.646860 7509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-0/wal
uuid: "c573d0f081194f1ebaaf440a0edb5074"
format_stamp: "Formatted at 2026-08-08 18:25:15 on dist-test-slave-5k54"
server_key: "f2a7fa5f70a86fdfa0e8196d8a11fafe"
server_key_iv: "7590036b47f987afc856a07d08544ff5"
server_key_version: "encryptionkey@0"
I20260808 18:25:17.647389 7509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:25:17.662840 7509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:25:17.666074 7509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:25:17.666275 7509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:25:17.666898 7509 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:25:17.668052 7567 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 18:25:17.673795 7509 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 18:25:17.673871 7509 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260808 18:25:17.673966 7509 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 18:25:17.675802 7509 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 18:25:17.675868 7509 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.002s
I20260808 18:25:17.676182 7567 tablet_bootstrap.cc:492] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074: Bootstrap starting.
I20260808 18:25:17.686726 7509 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.129:42583
I20260808 18:25:17.686964 7681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.129:42583 every 8 connection(s)
I20260808 18:25:17.687846 7509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
I20260808 18:25:17.695721 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 7509
I20260808 18:25:17.696213 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.130:38425
--local_ip_for_outbound_sockets=127.23.72.130
--tserver_master_addrs=127.23.72.190:37911
--webserver_port=39023
--webserver_interface=127.23.72.130
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab
--principal=kudu/127.23.72.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:38763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:25:17.698582 7567 log.cc:824] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074: Log is configured to *not* fsync() on all Append() calls
I20260808 18:25:17.705839 7567 tablet_bootstrap.cc:492] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 18:25:17.706205 7567 tablet_bootstrap.cc:492] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074: Bootstrap complete.
I20260808 18:25:17.707324 7567 ts_tablet_manager.cc:1397] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074: Time spent bootstrapping tablet: real 0.031s user 0.017s sys 0.010s
I20260808 18:25:17.709796 7567 raft_consensus.cc:348] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "447f81cd31b74dccbf6a0d3b113a81f4" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40735 } } peers { permanent_uuid: "477e6890969f4889a60eea8e71c7201f" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 38425 } } peers { permanent_uuid: "c573d0f081194f1ebaaf440a0edb5074" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 42583 } }
I20260808 18:25:17.710297 7567 raft_consensus.cc:729] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c573d0f081194f1ebaaf440a0edb5074, State: Initialized, Role: FOLLOWER
I20260808 18:25:17.710779 7567 consensus_queue.cc:260] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "447f81cd31b74dccbf6a0d3b113a81f4" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40735 } } peers { permanent_uuid: "477e6890969f4889a60eea8e71c7201f" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 38425 } } peers { permanent_uuid: "c573d0f081194f1ebaaf440a0edb5074" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 42583 } }
I20260808 18:25:17.711864 7567 ts_tablet_manager.cc:1428] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074: Time spent starting tablet: real 0.004s user 0.008s sys 0.000s
W20260808 18:25:17.737460 7684 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:25:17.689784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.129:54347)
0808 18:25:17.690299 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:17.690354 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:17.691221 (+ 867us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:17.736675 (+ 45454us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":158,"threads_started":1}
W20260808 18:25:17.738086 7682 heartbeater.cc:644] Failed to heartbeat to 127.23.72.190:37911 (0 consecutive failures): Network error: Failed to ping master at 127.23.72.190:37911: Client connection negotiation failed: client connection to 127.23.72.190:37911: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 18:25:17.738821 7684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:17.738570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.129:52739)
0808 18:25:17.738699 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:17.738769 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 18:25:17.739537 7684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:17.739297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.129:47885)
0808 18:25:17.739402 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:17.739467 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 18:25:17.739763 7682 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 18:25:17.821480 7685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:25:17.821743 7685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:25:17.821784 7685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:25:17.825925 7685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:25:17.825985 7685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:25:17.826061 7685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.130
I20260808 18:25:17.831336 7685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:38763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab
--principal=kudu/127.23.72.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.130:38425
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.72.130
--webserver_port=39023
--tserver_master_addrs=127.23.72.190:37911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.130
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:25:17.832430 7685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:25:17.833453 7685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:25:17.843313 7694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:17.843297 7693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:25:17.843353 7685 server_base.cc:1034] running on GCE node
W20260808 18:25:17.843283 7696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:25:17.843931 7685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:25:17.844630 7685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:25:17.845824 7685 hybrid_clock.cc:648] HybridClock initialized: now 1786213517845818 us; error 45 us; skew 500 ppm
Aug 08 18:25:17 dist-test-slave-5k54 krb5kdc[6294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213517, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:25:17.848861 7685 init.cc:377] Logged in from keytab as kudu/127.23.72.130@KRBTEST.COM (short username kudu)
I20260808 18:25:17.849762 7685 webserver.cc:460] Webserver started at http://127.23.72.130:39023/ using document root <none> and password file <none>
I20260808 18:25:17.850409 7685 fs_manager.cc:359] Metadata directory not provided
I20260808 18:25:17.850497 7685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:25:17.854359 7685 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:25:17.856544 7703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:17.857748 7685 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:25:17.857877 7685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-1/wal
uuid: "477e6890969f4889a60eea8e71c7201f"
format_stamp: "Formatted at 2026-08-08 18:25:15 on dist-test-slave-5k54"
server_key: "17346c4437bfba79f09bae0e10393e27"
server_key_iv: "a0b37598c0509654609f3e886bcfbe26"
server_key_version: "encryptionkey@0"
I20260808 18:25:17.858405 7685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:25:17.869656 7685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:25:17.872538 7685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:25:17.872733 7685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:25:17.873296 7685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:25:17.874343 7710 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260808 18:25:17.878108 7710 ts_tablet_manager.cc:1852] T b1581f1b34b4461e9aa174a0c0bd5b44 P 477e6890969f4889a60eea8e71c7201f: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260808 18:25:17.878497 7710 ts_tablet_manager.cc:1905] T b1581f1b34b4461e9aa174a0c0bd5b44 P 477e6890969f4889a60eea8e71c7201f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:25:17.882248 7710 ts_tablet_manager.cc:1918] T b1581f1b34b4461e9aa174a0c0bd5b44 P 477e6890969f4889a60eea8e71c7201f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:25:17.882359 7710 log.cc:1196] T b1581f1b34b4461e9aa174a0c0bd5b44 P 477e6890969f4889a60eea8e71c7201f: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-1/wal/wals/b1581f1b34b4461e9aa174a0c0bd5b44
I20260808 18:25:17.882695 7710 ts_tablet_manager.cc:1939] T b1581f1b34b4461e9aa174a0c0bd5b44 P 477e6890969f4889a60eea8e71c7201f: Deleting consensus metadata
I20260808 18:25:17.883025 7685 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260808 18:25:17.883113 7685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.009s user 0.000s sys 0.000s
I20260808 18:25:17.883193 7685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:25:17.883232 7685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:17.892881 7685 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.130:38425
I20260808 18:25:17.892939 7820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.130:38425 every 8 connection(s)
I20260808 18:25:17.893723 7685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
I20260808 18:25:17.894788 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 7685
I20260808 18:25:17.895254 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.131:40735
--local_ip_for_outbound_sockets=127.23.72.131
--tserver_master_addrs=127.23.72.190:37911
--webserver_port=40977
--webserver_interface=127.23.72.131
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab
--principal=kudu/127.23.72.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:38763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:25:17.897426 7823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:17.895830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.130:37017)
0808 18:25:17.896383 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:17.896525 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":124,"threads_started":1}
W20260808 18:25:17.897855 7821 heartbeater.cc:644] Failed to heartbeat to 127.23.72.190:37911 (0 consecutive failures): Network error: Failed to ping master at 127.23.72.190:37911: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
I20260808 18:25:17.898507 7823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:17.898262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.130:33121)
0808 18:25:17.898409 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:17.898455 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260808 18:25:17.899120 7823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:17.898946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.130:51331)
0808 18:25:17.899038 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:17.899077 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 18:25:17.899343 7821 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 18:25:18.008939 7824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:25:18.009208 7824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:25:18.009300 7824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:25:18.013777 7824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:25:18.013845 7824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:25:18.013942 7824 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.131
I20260808 18:25:18.019412 7824 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:38763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab
--principal=kudu/127.23.72.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.131:40735
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.72.131
--webserver_port=40977
--tserver_master_addrs=127.23.72.190:37911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7824
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.131
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:25:18.020478 7824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:25:18.021471 7824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:25:18.030973 7832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:25:18.031037 7824 server_base.cc:1034] running on GCE node
W20260808 18:25:18.031002 7830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:18.030992 7829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:25:18.031598 7824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:25:18.032229 7824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:25:18.033409 7824 hybrid_clock.cc:648] HybridClock initialized: now 1786213518033399 us; error 28 us; skew 500 ppm
Aug 08 18:25:18 dist-test-slave-5k54 krb5kdc[6294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213518, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:25:18.036181 7824 init.cc:377] Logged in from keytab as kudu/127.23.72.131@KRBTEST.COM (short username kudu)
I20260808 18:25:18.036995 7824 webserver.cc:460] Webserver started at http://127.23.72.131:40977/ using document root <none> and password file <none>
I20260808 18:25:18.037546 7824 fs_manager.cc:359] Metadata directory not provided
I20260808 18:25:18.037636 7824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:25:18.041392 7824 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 18:25:18.043351 7839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:18.044747 7824 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:25:18.044945 7824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-2/wal
uuid: "447f81cd31b74dccbf6a0d3b113a81f4"
format_stamp: "Formatted at 2026-08-08 18:25:15 on dist-test-slave-5k54"
server_key: "a2082b064639115d217cba86fcedb5ea"
server_key_iv: "b6a58bf6c75d31a6cf218a7c810d308a"
server_key_version: "encryptionkey@0"
I20260808 18:25:18.045490 7824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:25:18.059166 7824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:25:18.062357 7824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:25:18.062577 7824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:25:18.063243 7824 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:25:18.064110 7824 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:25:18.064183 7824 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:18.064256 7824 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:25:18.064307 7824 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:18.074028 7824 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.131:40735
I20260808 18:25:18.074033 7958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.131:40735 every 8 connection(s)
I20260808 18:25:18.075080 7824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
I20260808 18:25:18.079257 7961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:18.077316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.131:47749)
0808 18:25:18.077828 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:18.077953 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":118,"threads_started":1}
W20260808 18:25:18.079754 7959 heartbeater.cc:644] Failed to heartbeat to 127.23.72.190:37911 (0 consecutive failures): Network error: Failed to ping master at 127.23.72.190:37911: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
I20260808 18:25:18.080318 7961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:18.080113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.131:42573)
0808 18:25:18.080218 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:18.080265 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 18:25:18.080972 7961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:18.080776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.131:33601)
0808 18:25:18.080887 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:18.080928 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 18:25:18.081257 7959 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 18:25:18.082944 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 7824
I20260808 18:25:18.740789 7963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:18.740364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.129:43933)
0808 18:25:18.740614 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:18.740674 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":77,"threads_started":1}
I20260808 18:25:18.900218 7964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:18.899837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.130:50309)
0808 18:25:18.900079 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:18.900135 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":69,"threads_started":1}
I20260808 18:25:19.082170 7965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:19.081777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.131:33523)
0808 18:25:19.082018 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:19.082069 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":68,"threads_started":1}
I20260808 18:25:19.168920 7966 raft_consensus.cc:482] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:25:19.169138 7966 raft_consensus.cc:504] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "447f81cd31b74dccbf6a0d3b113a81f4" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40735 } } peers { permanent_uuid: "477e6890969f4889a60eea8e71c7201f" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 38425 } } peers { permanent_uuid: "c573d0f081194f1ebaaf440a0edb5074" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 42583 } }
I20260808 18:25:19.170414 7966 leader_election.cc:288] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 447f81cd31b74dccbf6a0d3b113a81f4 (127.23.72.131:40735), 477e6890969f4889a60eea8e71c7201f (127.23.72.130:38425)
Aug 08 18:25:19 dist-test-slave-5k54 krb5kdc[6294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213517, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for kudu/127.23.72.131@KRBTEST.COM
Aug 08 18:25:19 dist-test-slave-5k54 krb5kdc[6294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213517, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for kudu/127.23.72.130@KRBTEST.COM
I20260808 18:25:19.185128 7963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:19.170520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.131:40735 (local address 127.23.72.129:52013)
0808 18:25:19.170740 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:19.170757 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:19.170864 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:19.172682 (+ 1818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:19.172688 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:19.173280 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:25:19.173714 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:19.173728 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:19.175003 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:19.175006 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:19.175997 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:19.176007 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:19.176709 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:19.176712 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:19.176889 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:19.177581 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:25:19.177597 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:25:19.179508 (+ 1911us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:25:19.181997 (+ 2489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:19.182003 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:19.182018 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:19.182523 (+ 505us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:19.182969 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:19.182973 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:19.182975 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:19.183190 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:19.183962 (+ 772us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:25:19.183969 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:25:19.184343 (+ 374us) client_negotiation.cc:769] Sending connection context
0808 18:25:19.184563 (+ 220us) client_negotiation.cc:240] Negotiation successful
0808 18:25:19.184980 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":257,"spinlock_wait_cycles":40192}
I20260808 18:25:19.185209 7968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:19.170754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:52013 (local address 127.23.72.131:40735)
0808 18:25:19.171132 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:19.171159 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:19.171206 (+ 47us) server_negotiation.cc:407] Connection header received
0808 18:25:19.171962 (+ 756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:19.171968 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:19.172311 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:19.172625 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:25:19.173900 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:19.174853 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:19.176120 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:19.176576 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:19.176760 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:19.179721 (+ 2961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:25:19.179726 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:25:19.179736 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:25:19.179763 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:25:19.181853 (+ 2090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:19.182664 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:19.182671 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:19.182676 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:19.182828 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:19.183291 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:19.183295 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:19.183297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:19.183830 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:25:19.183978 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:19.184709 (+ 731us) server_negotiation.cc:299] Negotiation successful
0808 18:25:19.185016 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":139,"threads_started":1}
I20260808 18:25:19.185204 7969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:19.170853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:40285 (local address 127.23.72.130:38425)
0808 18:25:19.171214 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:19.171224 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:19.171258 (+ 34us) server_negotiation.cc:407] Connection header received
0808 18:25:19.172017 (+ 759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:19.172024 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:19.172324 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:19.172625 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:25:19.173901 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:19.174854 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:19.176121 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:19.176577 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:19.176750 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:19.179927 (+ 3177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:25:19.179934 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:25:19.179948 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:25:19.179982 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:25:19.181862 (+ 1880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:19.182664 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:19.182671 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:19.182676 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:19.182870 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:19.183292 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:19.183295 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:19.183297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:19.183868 (+ 571us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:25:19.183980 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:19.184709 (+ 729us) server_negotiation.cc:299] Negotiation successful
0808 18:25:19.185020 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":115,"threads_started":1}
I20260808 18:25:19.185127 7967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:19.170693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.130:38425 (local address 127.23.72.129:40285)
0808 18:25:19.171080 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:19.171094 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:19.171185 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:19.172683 (+ 1498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:19.172688 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:19.173280 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:25:19.173713 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:19.173728 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:19.175013 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:19.175017 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:19.176001 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:19.176010 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:19.176711 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:19.176715 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:19.176900 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:19.177628 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:25:19.177640 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:25:19.179734 (+ 2094us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:25:19.181997 (+ 2263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:19.182003 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:19.182018 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:19.182523 (+ 505us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:19.182969 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:19.182973 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:19.182975 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:19.183190 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:19.183963 (+ 773us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:25:19.183969 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:25:19.184343 (+ 374us) client_negotiation.cc:769] Sending connection context
0808 18:25:19.184563 (+ 220us) client_negotiation.cc:240] Negotiation successful
0808 18:25:19.184923 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"mutex_wait_us":203,"spinlock_wait_cycles":29440,"thread_start_us":77,"threads_started":1}
W20260808 18:25:19.187772 7564 leader_election.cc:341] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 477e6890969f4889a60eea8e71c7201f (127.23.72.130:38425): Not found: Tablet not found: b1581f1b34b4461e9aa174a0c0bd5b44
W20260808 18:25:19.187942 7563 leader_election.cc:341] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 447f81cd31b74dccbf6a0d3b113a81f4 (127.23.72.131:40735): Not found: Tablet not found: b1581f1b34b4461e9aa174a0c0bd5b44
I20260808 18:25:19.188016 7563 leader_election.cc:302] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c573d0f081194f1ebaaf440a0edb5074; no voters: 447f81cd31b74dccbf6a0d3b113a81f4, 477e6890969f4889a60eea8e71c7201f
I20260808 18:25:19.188285 7966 raft_consensus.cc:2726] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 18:25:19.741964 7970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:19.741562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.129:59429)
0808 18:25:19.741816 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:19.741869 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":77,"threads_started":1}
I20260808 18:25:19.901386 7971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:19.900973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.130:45313)
0808 18:25:19.901216 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:19.901273 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":77,"threads_started":1}
I20260808 18:25:20.083340 7972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:20.082938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.131:40833)
0808 18:25:20.083186 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:20.083244 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":76,"threads_started":1}
I20260808 18:25:20.743153 7973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:20.742730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.129:47389)
0808 18:25:20.743002 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:20.743057 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":116,"threads_started":1}
I20260808 18:25:20.902480 7974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:20.902098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.130:40205)
0808 18:25:20.902335 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:20.902390 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":75,"threads_started":1}
I20260808 18:25:21.084434 7975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:21.084022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.131:52713)
0808 18:25:21.084289 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:21.084340 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":96,"threads_started":1}
I20260808 18:25:21.340793 7976 raft_consensus.cc:482] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:25:21.340935 7976 raft_consensus.cc:504] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "447f81cd31b74dccbf6a0d3b113a81f4" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40735 } } peers { permanent_uuid: "477e6890969f4889a60eea8e71c7201f" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 38425 } } peers { permanent_uuid: "c573d0f081194f1ebaaf440a0edb5074" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 42583 } }
I20260808 18:25:21.341317 7976 leader_election.cc:288] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 447f81cd31b74dccbf6a0d3b113a81f4 (127.23.72.131:40735), 477e6890969f4889a60eea8e71c7201f (127.23.72.130:38425)
W20260808 18:25:21.342252 7564 leader_election.cc:341] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 477e6890969f4889a60eea8e71c7201f (127.23.72.130:38425): Not found: Tablet not found: b1581f1b34b4461e9aa174a0c0bd5b44
W20260808 18:25:21.342365 7563 leader_election.cc:341] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 447f81cd31b74dccbf6a0d3b113a81f4 (127.23.72.131:40735): Not found: Tablet not found: b1581f1b34b4461e9aa174a0c0bd5b44
I20260808 18:25:21.342474 7563 leader_election.cc:302] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c573d0f081194f1ebaaf440a0edb5074; no voters: 447f81cd31b74dccbf6a0d3b113a81f4, 477e6890969f4889a60eea8e71c7201f
I20260808 18:25:21.342631 7976 raft_consensus.cc:2726] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 18:25:21.744477 7977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:21.744061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.129:42767)
0808 18:25:21.744311 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:21.744378 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":75,"threads_started":1}
I20260808 18:25:21.903613 7978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:21.903159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.130:49613)
0808 18:25:21.903411 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:21.903509 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":93,"threads_started":1}
I20260808 18:25:22.085549 7979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:22.085144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.131:43487)
0808 18:25:22.085400 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:22.085456 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":74,"threads_started":1}
I20260808 18:25:22.745926 7980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:22.745415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.129:38233)
0808 18:25:22.745750 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:22.745814 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":79,"threads_started":1}
I20260808 18:25:22.904793 7981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:22.904382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.130:41961)
0808 18:25:22.904624 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:22.904681 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":73,"threads_started":1}
I20260808 18:25:23.086678 7982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:23.086268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.131:43565)
0808 18:25:23.086531 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:23.086586 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":95,"threads_started":1}
I20260808 18:25:23.747267 7983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:23.746829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.129:57289)
0808 18:25:23.747104 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:23.747168 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":78,"threads_started":1}
I20260808 18:25:23.906191 7984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:23.905615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.130:59165)
0808 18:25:23.905994 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:23.906067 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":114,"threads_started":1}
I20260808 18:25:24.087888 7985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:24.087407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.131:42433)
0808 18:25:24.087731 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:24.087786 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":95,"threads_started":1}
I20260808 18:25:24.598402 7986 raft_consensus.cc:482] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:25:24.598536 7986 raft_consensus.cc:504] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "447f81cd31b74dccbf6a0d3b113a81f4" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40735 } } peers { permanent_uuid: "477e6890969f4889a60eea8e71c7201f" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 38425 } } peers { permanent_uuid: "c573d0f081194f1ebaaf440a0edb5074" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 42583 } }
I20260808 18:25:24.598979 7986 leader_election.cc:288] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 447f81cd31b74dccbf6a0d3b113a81f4 (127.23.72.131:40735), 477e6890969f4889a60eea8e71c7201f (127.23.72.130:38425)
W20260808 18:25:24.599834 7563 leader_election.cc:341] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 447f81cd31b74dccbf6a0d3b113a81f4 (127.23.72.131:40735): Not found: Tablet not found: b1581f1b34b4461e9aa174a0c0bd5b44
W20260808 18:25:24.599932 7564 leader_election.cc:341] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 477e6890969f4889a60eea8e71c7201f (127.23.72.130:38425): Not found: Tablet not found: b1581f1b34b4461e9aa174a0c0bd5b44
I20260808 18:25:24.600000 7563 leader_election.cc:302] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c573d0f081194f1ebaaf440a0edb5074; no voters: 447f81cd31b74dccbf6a0d3b113a81f4, 477e6890969f4889a60eea8e71c7201f
I20260808 18:25:24.600152 7986 raft_consensus.cc:2726] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 18:25:24.748622 7987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:24.748162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.129:46353)
0808 18:25:24.748459 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:24.748523 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":94,"threads_started":1}
I20260808 18:25:24.907655 7988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:24.907069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.130:44195)
0808 18:25:24.907383 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:24.907526 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":77,"threads_started":1}
I20260808 18:25:25.089270 7989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:25.088658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.131:40113)
0808 18:25:25.089070 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:25.089150 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":99,"threads_started":1}
I20260808 18:25:25.749972 7990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:25.749546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.129:52631)
0808 18:25:25.749810 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:25.749869 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":85,"threads_started":1}
I20260808 18:25:25.908775 7991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:25.908374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.130:54389)
0808 18:25:25.908631 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:25.908686 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":96,"threads_started":1}
I20260808 18:25:26.090395 7992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:26.089976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.131:35873)
0808 18:25:26.090245 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:26.090300 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":77,"threads_started":1}
I20260808 18:25:26.751346 7993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:26.750876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.129:60779)
0808 18:25:26.751182 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:26.751247 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":99,"threads_started":1}
I20260808 18:25:26.909958 7994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:26.909530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.130:46275)
0808 18:25:26.909804 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:26.909860 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":98,"threads_started":1}
I20260808 18:25:27.091917 7995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:27.091303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.131:54367)
0808 18:25:27.091657 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:27.091717 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":185,"threads_started":1}
I20260808 18:25:27.655233 7996 raft_consensus.cc:482] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:25:27.655375 7996 raft_consensus.cc:504] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "447f81cd31b74dccbf6a0d3b113a81f4" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40735 } } peers { permanent_uuid: "477e6890969f4889a60eea8e71c7201f" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 38425 } } peers { permanent_uuid: "c573d0f081194f1ebaaf440a0edb5074" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 42583 } }
I20260808 18:25:27.655838 7996 leader_election.cc:288] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 447f81cd31b74dccbf6a0d3b113a81f4 (127.23.72.131:40735), 477e6890969f4889a60eea8e71c7201f (127.23.72.130:38425)
W20260808 18:25:27.656617 7563 leader_election.cc:341] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 447f81cd31b74dccbf6a0d3b113a81f4 (127.23.72.131:40735): Not found: Tablet not found: b1581f1b34b4461e9aa174a0c0bd5b44
W20260808 18:25:27.656728 7564 leader_election.cc:341] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 477e6890969f4889a60eea8e71c7201f (127.23.72.130:38425): Not found: Tablet not found: b1581f1b34b4461e9aa174a0c0bd5b44
I20260808 18:25:27.656788 7564 leader_election.cc:302] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c573d0f081194f1ebaaf440a0edb5074; no voters: 447f81cd31b74dccbf6a0d3b113a81f4, 477e6890969f4889a60eea8e71c7201f
I20260808 18:25:27.656937 7996 raft_consensus.cc:2726] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 18:25:27.752709 7997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:27.752299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.129:41899)
0808 18:25:27.752559 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:27.752613 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":74,"threads_started":1}
I20260808 18:25:27.911159 7998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:27.910723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.130:49073)
0808 18:25:27.911003 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:27.911061 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":74,"threads_started":1}
I20260808 18:25:28.093230 7999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:28.092829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.131:41847)
0808 18:25:28.093077 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:28.093135 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":76,"threads_started":1}
I20260808 18:25:28.753940 8000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:28.753455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.129:58955)
0808 18:25:28.753776 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:28.753841 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":83,"threads_started":1}
I20260808 18:25:28.912382 8001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:28.911969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.130:55181)
0808 18:25:28.912225 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:28.912278 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":72,"threads_started":1}
I20260808 18:25:29.094344 8002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:29.093942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.131:45223)
0808 18:25:29.094194 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:29.094252 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":80,"threads_started":1}
I20260808 18:25:29.755095 8003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:29.754694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.129:48741)
0808 18:25:29.754947 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:29.755001 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":76,"threads_started":1}
I20260808 18:25:29.913653 8004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:29.913194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.130:52273)
0808 18:25:29.913452 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:29.913509 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":85,"threads_started":1}
I20260808 18:25:30.095633 8005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:30.095111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.131:39571)
0808 18:25:30.095383 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:30.095506 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":82,"threads_started":1}
I20260808 18:25:30.112926 8006 raft_consensus.cc:482] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:25:30.113057 8006 raft_consensus.cc:504] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "447f81cd31b74dccbf6a0d3b113a81f4" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 40735 } } peers { permanent_uuid: "477e6890969f4889a60eea8e71c7201f" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 38425 } } peers { permanent_uuid: "c573d0f081194f1ebaaf440a0edb5074" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 42583 } }
I20260808 18:25:30.113548 8006 leader_election.cc:288] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 447f81cd31b74dccbf6a0d3b113a81f4 (127.23.72.131:40735), 477e6890969f4889a60eea8e71c7201f (127.23.72.130:38425)
W20260808 18:25:30.114351 7563 leader_election.cc:341] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 447f81cd31b74dccbf6a0d3b113a81f4 (127.23.72.131:40735): Not found: Tablet not found: b1581f1b34b4461e9aa174a0c0bd5b44
W20260808 18:25:30.114466 7564 leader_election.cc:341] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 477e6890969f4889a60eea8e71c7201f (127.23.72.130:38425): Not found: Tablet not found: b1581f1b34b4461e9aa174a0c0bd5b44
I20260808 18:25:30.114593 7564 leader_election.cc:302] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c573d0f081194f1ebaaf440a0edb5074; no voters: 447f81cd31b74dccbf6a0d3b113a81f4, 477e6890969f4889a60eea8e71c7201f
I20260808 18:25:30.114758 8006 raft_consensus.cc:2726] T b1581f1b34b4461e9aa174a0c0bd5b44 P c573d0f081194f1ebaaf440a0edb5074 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 18:25:30.756296 8007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:30.755884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.129:43667)
0808 18:25:30.756141 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:30.756198 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":85,"threads_started":1}
I20260808 18:25:30.915107 8008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:30.914673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.130:54073)
0808 18:25:30.914960 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:30.915017 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":86,"threads_started":1}
I20260808 18:25:31.096868 8009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:31.096395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.131:52093)
0808 18:25:31.096681 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:31.096742 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":91,"threads_started":1}
I20260808 18:25:31.757470 8010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:31.757050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.129:40783)
0808 18:25:31.757321 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:31.757376 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":93,"threads_started":1}
I20260808 18:25:31.916272 8011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:31.915845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.130:48941)
0808 18:25:31.916124 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:31.916177 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":91,"threads_started":1}
I20260808 18:25:32.097980 8012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:32.097574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.131:48977)
0808 18:25:32.097820 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:32.097884 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":80,"threads_started":1}
I20260808 18:25:32.758807 8014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:32.758412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.129:60995)
0808 18:25:32.758660 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:32.758713 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":80,"threads_started":1}
I20260808 18:25:32.917650 8015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:32.917235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.130:52191)
0808 18:25:32.917488 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:32.917541 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":75,"threads_started":1}
I20260808 18:25:33.091885 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 7509
I20260808 18:25:33.099465 8016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:33.098974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:37911 (local address 127.23.72.131:52285)
0808 18:25:33.099247 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:33.099305 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:37911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":90,"threads_started":1}
I20260808 18:25:33.100399 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 7685
I20260808 18:25:33.107343 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 7824
2026-08-08T18:25:33Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19746 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:25:33 dist-test-slave-5k54 krb5kdc[8019](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:25:33 dist-test-slave-5k54 krb5kdc[8019](info): set up 2 sockets
Aug 08 18:25:33 dist-test-slave-5k54 krb5kdc[8019](info): commencing operation
krb5kdc: starting...
W20260808 18:25:34.209282 23842 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Aug 08 18:25:34 dist-test-slave-5k54 krb5kdc[8019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213534, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:25:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:25:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.72.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
I20260808 18:25:34.339146 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:46191
--webserver_interface=127.23.72.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:45461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:46191
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:25:34.454772 8032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:25:34.455080 8032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:25:34.455179 8032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:25:34.459200 8032 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 18:25:34.459277 8032 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 18:25:34.459304 8032 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 18:25:34.459338 8032 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:25:34.459376 8032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:25:34.459414 8032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:25:34.459468 8032 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:25:34.459503 8032 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:25:34.464875 8032 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:45461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:46191
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:46191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.72.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:25:34.465921 8032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:25:34.466924 8032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:25:34.475705 8037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:34.475723 8039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:34.475718 8042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:25:34.475759 8032 server_base.cc:1034] running on GCE node
I20260808 18:25:34.476635 8032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:25:34.477785 8032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:25:34.478977 8032 hybrid_clock.cc:648] HybridClock initialized: now 1786213534478960 us; error 31 us; skew 500 ppm
Aug 08 18:25:34 dist-test-slave-5k54 krb5kdc[8019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213534, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:25:34.484993 8032 init.cc:377] Logged in from keytab as kudu/127.23.72.190@KRBTEST.COM (short username kudu)
I20260808 18:25:34.485939 8032 webserver.cc:460] Webserver started at http://127.23.72.190:42271/ using document root <none> and password file <none>
I20260808 18:25:34.486502 8032 fs_manager.cc:359] Metadata directory not provided
I20260808 18:25:34.486593 8032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:25:34.486824 8032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:25:34.489291 8032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "42c149b9c53a4e80a854d8db25ac438d"
format_stamp: "Formatted at 2026-08-08 18:25:34 on dist-test-slave-5k54"
server_key: "ffc482c5a6c05b21ee579622e101f4e0"
server_key_iv: "89e73e4faaa2d7598ff1a669c1d02846"
server_key_version: "encryptionkey@0"
I20260808 18:25:34.489953 8032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "42c149b9c53a4e80a854d8db25ac438d"
format_stamp: "Formatted at 2026-08-08 18:25:34 on dist-test-slave-5k54"
server_key: "ffc482c5a6c05b21ee579622e101f4e0"
server_key_iv: "89e73e4faaa2d7598ff1a669c1d02846"
server_key_version: "encryptionkey@0"
I20260808 18:25:34.494091 8032 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260808 18:25:34.496585 8049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:34.497843 8032 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:25:34.497995 8032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "42c149b9c53a4e80a854d8db25ac438d"
format_stamp: "Formatted at 2026-08-08 18:25:34 on dist-test-slave-5k54"
server_key: "ffc482c5a6c05b21ee579622e101f4e0"
server_key_iv: "89e73e4faaa2d7598ff1a669c1d02846"
server_key_version: "encryptionkey@0"
I20260808 18:25:34.498103 8032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:25:34.507637 8032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:25:34.510826 8032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:25:34.511030 8032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:25:34.518895 8032 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.190:46191
I20260808 18:25:34.519224 8104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.190:46191 every 8 connection(s)
I20260808 18:25:34.520251 8032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:25:34.523063 8105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:25:34.525898 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 8032
I20260808 18:25:34.526054 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:25:34.526371 23842 external_mini_cluster.cc:1428] Setting key d5eea8ef8cea710bc47dbc08cb2bdeca
I20260808 18:25:34.529685 8105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d: Bootstrap starting.
I20260808 18:25:34.532684 8105 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d: Neither blocks nor log segments found. Creating new log.
I20260808 18:25:34.533511 8105 log.cc:824] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d: Log is configured to *not* fsync() on all Append() calls
I20260808 18:25:34.535753 8105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d: No bootstrap required, opened a new log
Aug 08 18:25:34 dist-test-slave-5k54 krb5kdc[8019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213534, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:25:34.539695 8105 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c149b9c53a4e80a854d8db25ac438d" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 46191 } }
I20260808 18:25:34.539956 8105 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:25:34.540045 8105 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42c149b9c53a4e80a854d8db25ac438d, State: Initialized, Role: FOLLOWER
I20260808 18:25:34.540596 8105 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c149b9c53a4e80a854d8db25ac438d" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 46191 } }
I20260808 18:25:34.540791 8105 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:25:34.540932 8105 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:25:34.541081 8105 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:25:34.542086 8105 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c149b9c53a4e80a854d8db25ac438d" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 46191 } }
I20260808 18:25:34.542501 8105 leader_election.cc:302] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42c149b9c53a4e80a854d8db25ac438d; no voters:
I20260808 18:25:34.542934 8105 leader_election.cc:288] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:25:34.543011 8110 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:25:34.543219 8110 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [term 1 LEADER]: Becoming Leader. State: Replica: 42c149b9c53a4e80a854d8db25ac438d, State: Running, Role: LEADER
I20260808 18:25:34.543646 8110 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c149b9c53a4e80a854d8db25ac438d" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 46191 } }
I20260808 18:25:34.543721 8108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:34.527909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45706 (local address 127.23.72.190:46191)
0808 18:25:34.528389 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:34.528412 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:34.528459 (+ 47us) server_negotiation.cc:407] Connection header received
0808 18:25:34.529359 (+ 900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:34.529383 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:34.529855 (+ 472us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:34.530243 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:25:34.531060 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:34.532707 (+ 1647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:34.533647 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:34.534340 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:34.534524 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:34.537735 (+ 3211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:25:34.537741 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:25:34.537754 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:25:34.537781 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:25:34.540326 (+ 2545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:34.540913 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:34.540919 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:34.540925 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:34.541105 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:34.541423 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:34.541427 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:34.541428 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:34.541837 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:25:34.542067 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:34.542337 (+ 270us) server_negotiation.cc:299] Negotiation successful
0808 18:25:34.542669 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":106,"threads_started":1}
I20260808 18:25:34.544344 8105 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:25:34.544966 8110 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42c149b9c53a4e80a854d8db25ac438d. Latest consensus state: current_term: 1 leader_uuid: "42c149b9c53a4e80a854d8db25ac438d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c149b9c53a4e80a854d8db25ac438d" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 46191 } } }
I20260808 18:25:34.545094 8110 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [sys.catalog]: This master's current role is: LEADER
I20260808 18:25:34.545776 8111 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42c149b9c53a4e80a854d8db25ac438d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c149b9c53a4e80a854d8db25ac438d" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 46191 } } }
I20260808 18:25:34.546150 8111 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [sys.catalog]: This master's current role is: LEADER
I20260808 18:25:34.546538 8119 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:25:34.549600 8119 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:25:34.555183 8119 catalog_manager.cc:1347] Generated new cluster ID: cfbae75465cc4c53a9de1e6a0f1a7efa
I20260808 18:25:34.555256 8119 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:25:34.566897 8119 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:25:34.567921 8119 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:25:34.574604 8119 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d: Generated new TSK 0
I20260808 18:25:34.575274 8119 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.72.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab.
Entry for principal kudu/127.23.72.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab.
I20260808 18:25:34.607851 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.129:0
--local_ip_for_outbound_sockets=127.23.72.129
--webserver_interface=127.23.72.129
--webserver_port=0
--tserver_master_addrs=127.23.72.190:46191
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab
--principal=kudu/127.23.72.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:45461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:25:34.735615 8132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:25:34.735919 8132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:25:34.735994 8132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:25:34.740218 8132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:25:34.740296 8132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:25:34.740384 8132 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.129
I20260808 18:25:34.745807 8132 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:45461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab
--principal=kudu/127.23.72.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.72.129
--webserver_port=0
--tserver_master_addrs=127.23.72.190:46191
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8132
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.129
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:25:34.746851 8132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:25:34.747826 8132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:25:34.757409 8138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:34.757419 8137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:34.757418 8140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:25:34.757470 8132 server_base.cc:1034] running on GCE node
I20260808 18:25:34.758080 8132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:25:34.758719 8132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:25:34.759903 8132 hybrid_clock.cc:648] HybridClock initialized: now 1786213534759893 us; error 30 us; skew 500 ppm
Aug 08 18:25:34 dist-test-slave-5k54 krb5kdc[8019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213534, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:25:34.762917 8132 init.cc:377] Logged in from keytab as kudu/127.23.72.129@KRBTEST.COM (short username kudu)
I20260808 18:25:34.763829 8132 webserver.cc:460] Webserver started at http://127.23.72.129:45303/ using document root <none> and password file <none>
I20260808 18:25:34.764357 8132 fs_manager.cc:359] Metadata directory not provided
I20260808 18:25:34.764421 8132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:25:34.764583 8132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:25:34.766539 8132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-0/data/instance:
uuid: "69302204002a49cea7f3730f16da5a25"
format_stamp: "Formatted at 2026-08-08 18:25:34 on dist-test-slave-5k54"
server_key: "756f5900f5382ef341510d7ca5b9494f"
server_key_iv: "7f3f90e976bcfcb36899215563a6cfe9"
server_key_version: "encryptionkey@0"
I20260808 18:25:34.767048 8132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance:
uuid: "69302204002a49cea7f3730f16da5a25"
format_stamp: "Formatted at 2026-08-08 18:25:34 on dist-test-slave-5k54"
server_key: "756f5900f5382ef341510d7ca5b9494f"
server_key_iv: "7f3f90e976bcfcb36899215563a6cfe9"
server_key_version: "encryptionkey@0"
I20260808 18:25:34.770705 8132 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 18:25:34.773073 8147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:34.774302 8132 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:25:34.774394 8132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-0/wal
uuid: "69302204002a49cea7f3730f16da5a25"
format_stamp: "Formatted at 2026-08-08 18:25:34 on dist-test-slave-5k54"
server_key: "756f5900f5382ef341510d7ca5b9494f"
server_key_iv: "7f3f90e976bcfcb36899215563a6cfe9"
server_key_version: "encryptionkey@0"
I20260808 18:25:34.774500 8132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:25:34.783748 8132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:25:34.786370 8132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:25:34.786525 8132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:25:34.787006 8132 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:25:34.787817 8132 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:25:34.787863 8132 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:34.787899 8132 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:25:34.787915 8132 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:34.796778 8132 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.129:36963
I20260808 18:25:34.796810 8267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.129:36963 every 8 connection(s)
I20260808 18:25:34.797647 8132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
I20260808 18:25:34.804843 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 8132
I20260808 18:25:34.804996 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance
I20260808 18:25:34.805330 23842 external_mini_cluster.cc:1428] Setting key 5f45732adf1204d96b7b27568f936365
Aug 08 18:25:34 dist-test-slave-5k54 krb5kdc[8019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213534, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:25:34.811864 8108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:34.799527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:43443 (local address 127.23.72.190:46191)
0808 18:25:34.799674 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:34.799683 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:34.800453 (+ 770us) server_negotiation.cc:407] Connection header received
0808 18:25:34.801817 (+ 1364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:34.801820 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:34.801869 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:34.801947 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:25:34.803130 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:34.803734 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:34.804897 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:34.805276 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:34.805395 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:34.808037 (+ 2642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:25:34.808043 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:25:34.808049 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:25:34.808083 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:25:34.809648 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:34.810270 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:34.810273 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:34.810274 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:34.810350 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:34.810744 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:34.810747 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:34.810748 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:34.810918 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:25:34.810993 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:34.811636 (+ 643us) server_negotiation.cc:299] Negotiation successful
0808 18:25:34.811744 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 18:25:34.813100 8270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:34.799840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.129:43443)
0808 18:25:34.800297 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:34.800339 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:34.801576 (+ 1237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:34.802168 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:34.802175 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:34.802527 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:25:34.802998 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:34.803011 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:34.803845 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:34.803847 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:34.804722 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:34.804731 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:34.805446 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:34.805452 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:34.805612 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:34.806215 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:25:34.806230 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:25:34.807873 (+ 1643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:25:34.809769 (+ 1896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:34.809778 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:34.809794 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:34.810167 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:34.810451 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:34.810457 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:34.810462 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:34.810650 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:34.811023 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:25:34.811032 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:25:34.811345 (+ 313us) client_negotiation.cc:769] Sending connection context
0808 18:25:34.811583 (+ 238us) client_negotiation.cc:240] Negotiation successful
0808 18:25:34.811966 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":121,"threads_started":1}
I20260808 18:25:34.814519 8268 heartbeater.cc:347] Connected to a master server at 127.23.72.190:46191
I20260808 18:25:34.814781 8268 heartbeater.cc:464] Registering TS with master...
I20260808 18:25:34.815379 8268 heartbeater.cc:511] Master 127.23.72.190:46191 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.23.72.130@KRBTEST.COM; defaulting to no policy
I20260808 18:25:34.817184 8066 ts_manager.cc:194] Registered new tserver with Master: 69302204002a49cea7f3730f16da5a25 (127.23.72.129:36963)
I20260808 18:25:34.818351 8066 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.129@KRBTEST.COM'} at 127.23.72.129:43443
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab.
Entry for principal kudu/127.23.72.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab.
I20260808 18:25:34.836263 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.130:0
--local_ip_for_outbound_sockets=127.23.72.130
--webserver_interface=127.23.72.130
--webserver_port=0
--tserver_master_addrs=127.23.72.190:46191
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab
--principal=kudu/127.23.72.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:45461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:25:34.950718 8273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:25:34.951020 8273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:25:34.951098 8273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:25:34.955269 8273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:25:34.955356 8273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:25:34.955499 8273 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.130
I20260808 18:25:34.960621 8273 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:45461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab
--principal=kudu/127.23.72.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.72.130
--webserver_port=0
--tserver_master_addrs=127.23.72.190:46191
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8273
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.130
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:25:34.961642 8273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:25:34.962841 8273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:25:34.972220 8281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:34.972204 8278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:34.972205 8279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:25:34.972726 8273 server_base.cc:1034] running on GCE node
I20260808 18:25:34.973097 8273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:25:34.973706 8273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:25:34.974869 8273 hybrid_clock.cc:648] HybridClock initialized: now 1786213534974852 us; error 29 us; skew 500 ppm
Aug 08 18:25:34 dist-test-slave-5k54 krb5kdc[8019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213534, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:25:34.977726 8273 init.cc:377] Logged in from keytab as kudu/127.23.72.130@KRBTEST.COM (short username kudu)
I20260808 18:25:34.978629 8273 webserver.cc:460] Webserver started at http://127.23.72.130:38803/ using document root <none> and password file <none>
I20260808 18:25:34.979151 8273 fs_manager.cc:359] Metadata directory not provided
I20260808 18:25:34.979208 8273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:25:34.979370 8273 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:25:34.981359 8273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-1/data/instance:
uuid: "802eb57061d9457e817c3e700330b143"
format_stamp: "Formatted at 2026-08-08 18:25:34 on dist-test-slave-5k54"
server_key: "27a71935d215f18387c4d896c480cbbc"
server_key_iv: "715815ef6fedf61ff01271c330ae925d"
server_key_version: "encryptionkey@0"
I20260808 18:25:34.981861 8273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance:
uuid: "802eb57061d9457e817c3e700330b143"
format_stamp: "Formatted at 2026-08-08 18:25:34 on dist-test-slave-5k54"
server_key: "27a71935d215f18387c4d896c480cbbc"
server_key_iv: "715815ef6fedf61ff01271c330ae925d"
server_key_version: "encryptionkey@0"
I20260808 18:25:34.985407 8273 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 18:25:34.987694 8289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:34.988866 8273 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 18:25:34.988988 8273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-1/wal
uuid: "802eb57061d9457e817c3e700330b143"
format_stamp: "Formatted at 2026-08-08 18:25:34 on dist-test-slave-5k54"
server_key: "27a71935d215f18387c4d896c480cbbc"
server_key_iv: "715815ef6fedf61ff01271c330ae925d"
server_key_version: "encryptionkey@0"
I20260808 18:25:34.989090 8273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:25:35.007671 8273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:25:35.010984 8273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:25:35.011190 8273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:25:35.011861 8273 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:25:35.012658 8273 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:25:35.012706 8273 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:35.012774 8273 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:25:35.012816 8273 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:35.021795 8273 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.130:34819
I20260808 18:25:35.021819 8411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.130:34819 every 8 connection(s)
I20260808 18:25:35.022658 8273 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
I20260808 18:25:35.023167 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 8273
I20260808 18:25:35.023308 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance
I20260808 18:25:35.023653 23842 external_mini_cluster.cc:1428] Setting key 0d8d331ff83fdba9adeef2bceeaae196
Aug 08 18:25:35 dist-test-slave-5k54 krb5kdc[8019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213534, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.130@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.72.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.131@KRBTEST.COM" created.
I20260808 18:25:35.038854 8108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:35.024650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:40765 (local address 127.23.72.190:46191)
0808 18:25:35.024788 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:35.024792 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:35.025885 (+ 1093us) server_negotiation.cc:407] Connection header received
0808 18:25:35.027921 (+ 2036us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:35.027926 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:35.027991 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:35.028087 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:25:35.029290 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:35.029919 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:35.031833 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:35.032059 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:35.032150 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:35.035049 (+ 2899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:25:35.035052 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:25:35.035054 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:25:35.035078 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:25:35.036712 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:35.037256 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:35.037261 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:35.037263 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:35.037351 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:35.037699 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:35.037704 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:35.037706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:35.037898 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:25:35.038006 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:35.038538 (+ 532us) server_negotiation.cc:299] Negotiation successful
0808 18:25:35.038670 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 18:25:35.039675 8414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:35.025135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.130:40765)
0808 18:25:35.025707 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:35.025762 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:35.027660 (+ 1898us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:35.028248 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:35.028255 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:35.028614 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:25:35.029130 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:35.029154 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:35.030057 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:35.030064 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:35.031697 (+ 1633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:35.031712 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:35.032234 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:35.032242 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:35.032500 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:35.033141 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:25:35.033167 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:25:35.034877 (+ 1710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:25:35.036870 (+ 1993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:35.036875 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:35.036889 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:35.037149 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:35.037452 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:35.037455 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:35.037457 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:35.037595 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:35.038007 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:25:35.038014 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:25:35.038250 (+ 236us) client_negotiation.cc:769] Sending connection context
0808 18:25:35.038477 (+ 227us) client_negotiation.cc:240] Negotiation successful
0808 18:25:35.038804 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":143,"threads_started":1}
I20260808 18:25:35.040874 8412 heartbeater.cc:347] Connected to a master server at 127.23.72.190:46191
I20260808 18:25:35.041119 8412 heartbeater.cc:464] Registering TS with master...
I20260808 18:25:35.041591 8412 heartbeater.cc:511] Master 127.23.72.190:46191 requested a full tablet report, sending...
I20260808 18:25:35.042683 8066 ts_manager.cc:194] Registered new tserver with Master: 802eb57061d9457e817c3e700330b143 (127.23.72.130:34819)
I20260808 18:25:35.043355 8066 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.130@KRBTEST.COM'} at 127.23.72.130:40765
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab.
Entry for principal kudu/127.23.72.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab.
I20260808 18:25:35.056880 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.131:0
--local_ip_for_outbound_sockets=127.23.72.131
--webserver_interface=127.23.72.131
--webserver_port=0
--tserver_master_addrs=127.23.72.190:46191
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab
--principal=kudu/127.23.72.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:45461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:25:35.168985 8417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:25:35.169224 8417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:25:35.169265 8417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:25:35.173122 8417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:25:35.173168 8417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:25:35.173224 8417 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.131
I20260808 18:25:35.178314 8417 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:45461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab
--principal=kudu/127.23.72.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.72.131
--webserver_port=0
--tserver_master_addrs=127.23.72.190:46191
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8417
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.131
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:25:35.179293 8417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:25:35.180302 8417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:25:35.189695 8423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:35.189738 8425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:35.189735 8422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:25:35.189954 8417 server_base.cc:1034] running on GCE node
I20260808 18:25:35.190411 8417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:25:35.191025 8417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:25:35.192209 8417 hybrid_clock.cc:648] HybridClock initialized: now 1786213535192196 us; error 24 us; skew 500 ppm
Aug 08 18:25:35 dist-test-slave-5k54 krb5kdc[8019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213535, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:25:35.194926 8417 init.cc:377] Logged in from keytab as kudu/127.23.72.131@KRBTEST.COM (short username kudu)
I20260808 18:25:35.195848 8417 webserver.cc:460] Webserver started at http://127.23.72.131:33835/ using document root <none> and password file <none>
I20260808 18:25:35.196412 8417 fs_manager.cc:359] Metadata directory not provided
I20260808 18:25:35.196506 8417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:25:35.196733 8417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:25:35.198688 8417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-2/data/instance:
uuid: "0df99bbbc2c14b959b7f0a90656988af"
format_stamp: "Formatted at 2026-08-08 18:25:35 on dist-test-slave-5k54"
server_key: "66d1bf5677d875d05f11dce131dc72bc"
server_key_iv: "8428f367a03d10657fdf1b3da9a8eb69"
server_key_version: "encryptionkey@0"
I20260808 18:25:35.199311 8417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance:
uuid: "0df99bbbc2c14b959b7f0a90656988af"
format_stamp: "Formatted at 2026-08-08 18:25:35 on dist-test-slave-5k54"
server_key: "66d1bf5677d875d05f11dce131dc72bc"
server_key_iv: "8428f367a03d10657fdf1b3da9a8eb69"
server_key_version: "encryptionkey@0"
I20260808 18:25:35.202982 8417 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 18:25:35.205394 8432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:35.206593 8417 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 18:25:35.206734 8417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-2/wal
uuid: "0df99bbbc2c14b959b7f0a90656988af"
format_stamp: "Formatted at 2026-08-08 18:25:35 on dist-test-slave-5k54"
server_key: "66d1bf5677d875d05f11dce131dc72bc"
server_key_iv: "8428f367a03d10657fdf1b3da9a8eb69"
server_key_version: "encryptionkey@0"
I20260808 18:25:35.206873 8417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:25:35.249423 8417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:25:35.252513 8417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:25:35.252723 8417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:25:35.253357 8417 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:25:35.254215 8417 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:25:35.254288 8417 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:35.254369 8417 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:25:35.254412 8417 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:35.263772 8417 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.131:34733
I20260808 18:25:35.263792 8554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.131:34733 every 8 connection(s)
I20260808 18:25:35.264734 8417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
I20260808 18:25:35.274194 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 8417
I20260808 18:25:35.274304 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance
I20260808 18:25:35.274614 23842 external_mini_cluster.cc:1428] Setting key 4cfb957c5df25ffa753bf6cb1bf65896
Aug 08 18:25:35 dist-test-slave-5k54 krb5kdc[8019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213535, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.131@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:25:35.279564 8108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:35.266567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:54717 (local address 127.23.72.190:46191)
0808 18:25:35.266723 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:35.266726 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:35.267638 (+ 912us) server_negotiation.cc:407] Connection header received
0808 18:25:35.268919 (+ 1281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:35.268922 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:35.268967 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:35.269043 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:25:35.270230 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:35.270846 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:35.272180 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:35.272429 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:35.272523 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:35.275535 (+ 3012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:25:35.275544 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:25:35.275546 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:25:35.275572 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:25:35.277196 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:35.277834 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:35.277840 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:35.277844 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:35.277949 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:35.278334 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:35.278337 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:35.278339 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:35.278537 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:25:35.278658 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:35.279204 (+ 546us) server_negotiation.cc:299] Negotiation successful
0808 18:25:35.279347 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 18:25:35.280383 8557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:35.266989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.131:54717)
0808 18:25:35.267491 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:35.267533 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:35.268727 (+ 1194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:35.269203 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:35.269210 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:35.269583 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:25:35.270060 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:35.270073 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:35.270990 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:35.270992 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:35.272060 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:35.272069 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:35.272590 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:35.272595 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:35.272770 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:35.273414 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:25:35.273435 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:25:35.275312 (+ 1877us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:25:35.277390 (+ 2078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:35.277397 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:35.277413 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:35.277705 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:35.278057 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:35.278061 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:35.278063 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:35.278221 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:35.278656 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:25:35.278664 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:25:35.278944 (+ 280us) client_negotiation.cc:769] Sending connection context
0808 18:25:35.279141 (+ 197us) client_negotiation.cc:240] Negotiation successful
0808 18:25:35.279521 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":113,"threads_started":1}
I20260808 18:25:35.281536 8555 heartbeater.cc:347] Connected to a master server at 127.23.72.190:46191
I20260808 18:25:35.281774 8555 heartbeater.cc:464] Registering TS with master...
I20260808 18:25:35.282277 8555 heartbeater.cc:511] Master 127.23.72.190:46191 requested a full tablet report, sending...
I20260808 18:25:35.283325 8066 ts_manager.cc:194] Registered new tserver with Master: 0df99bbbc2c14b959b7f0a90656988af (127.23.72.131:34733)
I20260808 18:25:35.284121 8066 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.131@KRBTEST.COM'} at 127.23.72.131:54717
I20260808 18:25:35.288618 23842 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:25:35.298230 8108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:35.290605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45716 (local address 127.23.72.190:46191)
0808 18:25:35.290750 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:35.290755 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:35.290977 (+ 222us) server_negotiation.cc:407] Connection header received
0808 18:25:35.291071 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:35.291074 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:35.291114 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:35.291181 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:25:35.291978 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:35.292564 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:35.293503 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:35.293716 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:35.293798 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:35.295134 (+ 1336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:25:35.295137 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:25:35.295139 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:25:35.295161 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:25:35.296684 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:35.297093 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:35.297095 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:35.297097 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:35.297171 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:35.297406 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:35.297408 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:35.297409 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:35.297617 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:25:35.297721 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:35.297931 (+ 210us) server_negotiation.cc:299] Negotiation successful
0808 18:25:35.298026 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 18:25:35.301293 8066 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45716:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:25:35.304535 8066 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:25:35.318668 8567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:35.312923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:36963 (local address 127.0.0.1:55684)
0808 18:25:35.313241 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:35.313286 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:35.313448 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:35.314069 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:35.314075 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:35.314101 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:25:35.314243 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:35.314252 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:35.315643 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:35.315646 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:35.317001 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:35.317010 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:35.318099 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:35.318104 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:35.318224 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:35.318310 (+ 86us) client_negotiation.cc:769] Sending connection context
0808 18:25:35.318423 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 18:25:35.318545 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":29,"thread_start_us":81,"threads_started":1}
I20260808 18:25:35.318711 8568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:35.313011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55684 (local address 127.23.72.129:36963)
0808 18:25:35.313276 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:35.313294 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:35.313362 (+ 68us) server_negotiation.cc:407] Connection header received
0808 18:25:35.313665 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:35.313674 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:35.313905 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:35.314067 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:25:35.314389 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:35.315553 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:35.317130 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:35.317961 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:35.318091 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:35.318248 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:35.318469 (+ 221us) server_negotiation.cc:299] Negotiation successful
0808 18:25:35.318563 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":100,"threads_started":1}
I20260808 18:25:35.319119 8571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:35.313627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.130:34819 (local address 127.0.0.1:41602)
0808 18:25:35.314361 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:35.314375 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:35.314455 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:35.315093 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:35.315096 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:35.315114 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:25:35.315199 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:35.315204 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:35.316294 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:35.316299 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:35.317346 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:35.317355 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:35.318158 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:35.318161 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:35.318263 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:35.318866 (+ 603us) client_negotiation.cc:769] Sending connection context
0808 18:25:35.318937 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 18:25:35.318997 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"mutex_wait_us":556,"thread_start_us":99,"threads_started":1}
I20260808 18:25:35.319588 8570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:35.313783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41602 (local address 127.23.72.130:34819)
0808 18:25:35.314690 (+ 907us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:35.314702 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:35.314723 (+ 21us) server_negotiation.cc:407] Connection header received
0808 18:25:35.314789 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:35.314794 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:35.314962 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:35.315107 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:25:35.315293 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:35.316162 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:35.317465 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:35.318008 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:35.318312 (+ 304us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:35.318465 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:35.319311 (+ 846us) server_negotiation.cc:299] Negotiation successful
0808 18:25:35.319409 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":806,"thread_start_us":110,"threads_started":1}
I20260808 18:25:35.319617 8569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:35.313978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.131:34733 (local address 127.0.0.1:60018)
0808 18:25:35.314759 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:35.314777 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:35.314893 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:35.315338 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:35.315341 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:35.315357 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:25:35.315473 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:35.315480 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:35.316872 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:35.316875 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:35.318009 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:35.318022 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:35.319223 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:35.319226 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:35.319323 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:35.319344 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 18:25:35.319414 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 18:25:35.319504 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":719,"thread_start_us":91,"threads_started":1}
I20260808 18:25:35.319999 8572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:35.314376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60018 (local address 127.23.72.131:34733)
0808 18:25:35.314695 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:35.314707 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:35.314867 (+ 160us) server_negotiation.cc:407] Connection header received
0808 18:25:35.315000 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:35.315007 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:35.315205 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:35.315364 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:25:35.315568 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:35.316724 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:35.318145 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:35.318931 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:35.319034 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:35.319160 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:35.319773 (+ 613us) server_negotiation.cc:299] Negotiation successful
0808 18:25:35.319870 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":132,"threads_started":1}
I20260808 18:25:35.321578 8199 tablet_service.cc:1467] Processing CreateTablet for tablet fcb6c5d1d07843388690dd522e6f18bc (DEFAULT_TABLE table=test-table [id=10fc89082ea643b1adcc72bedbec4575]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:25:35.322582 8199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcb6c5d1d07843388690dd522e6f18bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:25:35.325917 8485 tablet_service.cc:1467] Processing CreateTablet for tablet fcb6c5d1d07843388690dd522e6f18bc (DEFAULT_TABLE table=test-table [id=10fc89082ea643b1adcc72bedbec4575]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:25:35.326330 8340 tablet_service.cc:1467] Processing CreateTablet for tablet fcb6c5d1d07843388690dd522e6f18bc (DEFAULT_TABLE table=test-table [id=10fc89082ea643b1adcc72bedbec4575]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:25:35.326862 8485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcb6c5d1d07843388690dd522e6f18bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:25:35.327313 8340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcb6c5d1d07843388690dd522e6f18bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:25:35.329918 8573 tablet_bootstrap.cc:492] T fcb6c5d1d07843388690dd522e6f18bc P 69302204002a49cea7f3730f16da5a25: Bootstrap starting.
I20260808 18:25:35.332329 8573 tablet_bootstrap.cc:654] T fcb6c5d1d07843388690dd522e6f18bc P 69302204002a49cea7f3730f16da5a25: Neither blocks nor log segments found. Creating new log.
I20260808 18:25:35.333220 8573 log.cc:824] T fcb6c5d1d07843388690dd522e6f18bc P 69302204002a49cea7f3730f16da5a25: Log is configured to *not* fsync() on all Append() calls
I20260808 18:25:35.333561 8575 tablet_bootstrap.cc:492] T fcb6c5d1d07843388690dd522e6f18bc P 802eb57061d9457e817c3e700330b143: Bootstrap starting.
I20260808 18:25:35.334481 8577 tablet_bootstrap.cc:492] T fcb6c5d1d07843388690dd522e6f18bc P 0df99bbbc2c14b959b7f0a90656988af: Bootstrap starting.
I20260808 18:25:35.335660 8573 tablet_bootstrap.cc:492] T fcb6c5d1d07843388690dd522e6f18bc P 69302204002a49cea7f3730f16da5a25: No bootstrap required, opened a new log
I20260808 18:25:35.335873 8573 ts_tablet_manager.cc:1397] T fcb6c5d1d07843388690dd522e6f18bc P 69302204002a49cea7f3730f16da5a25: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 18:25:35.336211 8575 tablet_bootstrap.cc:654] T fcb6c5d1d07843388690dd522e6f18bc P 802eb57061d9457e817c3e700330b143: Neither blocks nor log segments found. Creating new log.
I20260808 18:25:35.336617 8577 tablet_bootstrap.cc:654] T fcb6c5d1d07843388690dd522e6f18bc P 0df99bbbc2c14b959b7f0a90656988af: Neither blocks nor log segments found. Creating new log.
I20260808 18:25:35.337095 8575 log.cc:824] T fcb6c5d1d07843388690dd522e6f18bc P 802eb57061d9457e817c3e700330b143: Log is configured to *not* fsync() on all Append() calls
I20260808 18:25:35.337826 8577 log.cc:824] T fcb6c5d1d07843388690dd522e6f18bc P 0df99bbbc2c14b959b7f0a90656988af: Log is configured to *not* fsync() on all Append() calls
I20260808 18:25:35.339149 8575 tablet_bootstrap.cc:492] T fcb6c5d1d07843388690dd522e6f18bc P 802eb57061d9457e817c3e700330b143: No bootstrap required, opened a new log
I20260808 18:25:35.339339 8575 ts_tablet_manager.cc:1397] T fcb6c5d1d07843388690dd522e6f18bc P 802eb57061d9457e817c3e700330b143: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 18:25:35.339787 8577 tablet_bootstrap.cc:492] T fcb6c5d1d07843388690dd522e6f18bc P 0df99bbbc2c14b959b7f0a90656988af: No bootstrap required, opened a new log
I20260808 18:25:35.339962 8577 ts_tablet_manager.cc:1397] T fcb6c5d1d07843388690dd522e6f18bc P 0df99bbbc2c14b959b7f0a90656988af: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 18:25:35.339824 8573 raft_consensus.cc:348] T fcb6c5d1d07843388690dd522e6f18bc P 69302204002a49cea7f3730f16da5a25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69302204002a49cea7f3730f16da5a25" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 36963 } } peers { permanent_uuid: "802eb57061d9457e817c3e700330b143" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 34819 } } peers { permanent_uuid: "0df99bbbc2c14b959b7f0a90656988af" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 34733 } }
I20260808 18:25:35.340147 8573 raft_consensus.cc:374] T fcb6c5d1d07843388690dd522e6f18bc P 69302204002a49cea7f3730f16da5a25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:25:35.340221 8573 raft_consensus.cc:729] T fcb6c5d1d07843388690dd522e6f18bc P 69302204002a49cea7f3730f16da5a25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69302204002a49cea7f3730f16da5a25, State: Initialized, Role: FOLLOWER
I20260808 18:25:35.340669 8573 consensus_queue.cc:260] T fcb6c5d1d07843388690dd522e6f18bc P 69302204002a49cea7f3730f16da5a25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69302204002a49cea7f3730f16da5a25" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 36963 } } peers { permanent_uuid: "802eb57061d9457e817c3e700330b143" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 34819 } } peers { permanent_uuid: "0df99bbbc2c14b959b7f0a90656988af" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 34733 } }
I20260808 18:25:35.341655 8573 ts_tablet_manager.cc:1428] T fcb6c5d1d07843388690dd522e6f18bc P 69302204002a49cea7f3730f16da5a25: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 18:25:35.342049 8268 heartbeater.cc:503] Master 127.23.72.190:46191 was elected leader, sending a full tablet report...
I20260808 18:25:35.343228 8575 raft_consensus.cc:348] T fcb6c5d1d07843388690dd522e6f18bc P 802eb57061d9457e817c3e700330b143 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69302204002a49cea7f3730f16da5a25" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 36963 } } peers { permanent_uuid: "802eb57061d9457e817c3e700330b143" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 34819 } } peers { permanent_uuid: "0df99bbbc2c14b959b7f0a90656988af" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 34733 } }
I20260808 18:25:35.343515 8575 raft_consensus.cc:374] T fcb6c5d1d07843388690dd522e6f18bc P 802eb57061d9457e817c3e700330b143 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:25:35.343601 8575 raft_consensus.cc:729] T fcb6c5d1d07843388690dd522e6f18bc P 802eb57061d9457e817c3e700330b143 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 802eb57061d9457e817c3e700330b143, State: Initialized, Role: FOLLOWER
I20260808 18:25:35.343657 8577 raft_consensus.cc:348] T fcb6c5d1d07843388690dd522e6f18bc P 0df99bbbc2c14b959b7f0a90656988af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69302204002a49cea7f3730f16da5a25" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 36963 } } peers { permanent_uuid: "802eb57061d9457e817c3e700330b143" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 34819 } } peers { permanent_uuid: "0df99bbbc2c14b959b7f0a90656988af" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 34733 } }
I20260808 18:25:35.343918 8577 raft_consensus.cc:374] T fcb6c5d1d07843388690dd522e6f18bc P 0df99bbbc2c14b959b7f0a90656988af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:25:35.343974 8577 raft_consensus.cc:729] T fcb6c5d1d07843388690dd522e6f18bc P 0df99bbbc2c14b959b7f0a90656988af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0df99bbbc2c14b959b7f0a90656988af, State: Initialized, Role: FOLLOWER
I20260808 18:25:35.344071 8575 consensus_queue.cc:260] T fcb6c5d1d07843388690dd522e6f18bc P 802eb57061d9457e817c3e700330b143 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69302204002a49cea7f3730f16da5a25" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 36963 } } peers { permanent_uuid: "802eb57061d9457e817c3e700330b143" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 34819 } } peers { permanent_uuid: "0df99bbbc2c14b959b7f0a90656988af" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 34733 } }
I20260808 18:25:35.344368 8577 consensus_queue.cc:260] T fcb6c5d1d07843388690dd522e6f18bc P 0df99bbbc2c14b959b7f0a90656988af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69302204002a49cea7f3730f16da5a25" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 36963 } } peers { permanent_uuid: "802eb57061d9457e817c3e700330b143" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 34819 } } peers { permanent_uuid: "0df99bbbc2c14b959b7f0a90656988af" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 34733 } }
I20260808 18:25:35.344904 8412 heartbeater.cc:503] Master 127.23.72.190:46191 was elected leader, sending a full tablet report...
I20260808 18:25:35.345122 8575 ts_tablet_manager.cc:1428] T fcb6c5d1d07843388690dd522e6f18bc P 802eb57061d9457e817c3e700330b143: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 18:25:35.345310 8577 ts_tablet_manager.cc:1428] T fcb6c5d1d07843388690dd522e6f18bc P 0df99bbbc2c14b959b7f0a90656988af: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 18:25:35.345649 8555 heartbeater.cc:503] Master 127.23.72.190:46191 was elected leader, sending a full tablet report...
I20260808 18:25:35.355284 8581 raft_consensus.cc:482] T fcb6c5d1d07843388690dd522e6f18bc P 802eb57061d9457e817c3e700330b143 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:25:35.355508 8581 raft_consensus.cc:504] T fcb6c5d1d07843388690dd522e6f18bc P 802eb57061d9457e817c3e700330b143 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69302204002a49cea7f3730f16da5a25" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 36963 } } peers { permanent_uuid: "802eb57061d9457e817c3e700330b143" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 34819 } } peers { permanent_uuid: "0df99bbbc2c14b959b7f0a90656988af" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 34733 } }
I20260808 18:25:35.356487 8581 leader_election.cc:288] T fcb6c5d1d07843388690dd522e6f18bc P 802eb57061d9457e817c3e700330b143 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 69302204002a49cea7f3730f16da5a25 (127.23.72.129:36963), 0df99bbbc2c14b959b7f0a90656988af (127.23.72.131:34733)
I20260808 18:25:35.360723 8568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:35.356850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:34985 (local address 127.23.72.129:36963)
0808 18:25:35.357011 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:35.357017 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:35.357033 (+ 16us) server_negotiation.cc:407] Connection header received
0808 18:25:35.357084 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:35.357088 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:35.357136 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:35.357228 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:25:35.357615 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:35.358262 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:35.359348 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:35.359961 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:35.360094 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:35.360146 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:35.360527 (+ 381us) server_negotiation.cc:299] Negotiation successful
0808 18:25:35.360601 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260808 18:25:35.360723 8572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:35.357113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:56505 (local address 127.23.72.131:34733)
0808 18:25:35.357251 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:35.357255 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:35.357423 (+ 168us) server_negotiation.cc:407] Connection header received
0808 18:25:35.357522 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:35.357526 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:35.357582 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:35.357655 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:25:35.357932 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:35.358443 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:35.359512 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:35.360072 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:35.360193 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:35.360242 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:35.360527 (+ 285us) server_negotiation.cc:299] Negotiation successful
0808 18:25:35.360603 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 18:25:35.360770 8414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:35.356759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:36963 (local address 127.23.72.130:34985)
0808 18:25:35.356875 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:35.356891 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:35.357002 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:35.357321 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:35.357324 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:35.357349 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:25:35.357500 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:35.357507 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:35.358366 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:35.358376 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:35.359212 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:35.359221 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:35.360111 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:35.360114 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:35.360230 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:35.360402 (+ 172us) client_negotiation.cc:769] Sending connection context
0808 18:25:35.360485 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 18:25:35.360607 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":130}
I20260808 18:25:35.360778 8583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:35.357036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.131:34733 (local address 127.23.72.130:56505)
0808 18:25:35.357333 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:35.357345 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:35.357438 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:35.357682 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:35.357684 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:35.357701 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:25:35.357810 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:35.357816 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:35.358552 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:35.358554 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:35.359375 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:35.359386 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:35.360223 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:35.360226 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:35.360401 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:35.360419 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 18:25:35.360485 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 18:25:35.360604 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"mutex_wait_us":66,"thread_start_us":115,"threads_started":1}
I20260808 18:25:35.361501 8220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb6c5d1d07843388690dd522e6f18bc" candidate_uuid: "802eb57061d9457e817c3e700330b143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69302204002a49cea7f3730f16da5a25" is_pre_election: true
I20260808 18:25:35.361579 8505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb6c5d1d07843388690dd522e6f18bc" candidate_uuid: "802eb57061d9457e817c3e700330b143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0df99bbbc2c14b959b7f0a90656988af" is_pre_election: true
I20260808 18:25:35.361814 8220 raft_consensus.cc:2454] T fcb6c5d1d07843388690dd522e6f18bc P 69302204002a49cea7f3730f16da5a25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 802eb57061d9457e817c3e700330b143 in term 0.
I20260808 18:25:35.361855 8505 raft_consensus.cc:2454] T fcb6c5d1d07843388690dd522e6f18bc P 0df99bbbc2c14b959b7f0a90656988af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 802eb57061d9457e817c3e700330b143 in term 0.
I20260808 18:25:35.362349 8292 leader_election.cc:302] T fcb6c5d1d07843388690dd522e6f18bc P 802eb57061d9457e817c3e700330b143 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0df99bbbc2c14b959b7f0a90656988af, 802eb57061d9457e817c3e700330b143; no voters:
I20260808 18:25:35.362635 8581 raft_consensus.cc:2781] T fcb6c5d1d07843388690dd522e6f18bc P 802eb57061d9457e817c3e700330b143 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:25:35.362710 8581 raft_consensus.cc:482] T fcb6c5d1d07843388690dd522e6f18bc P 802eb57061d9457e817c3e700330b143 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:25:35.362758 8581 raft_consensus.cc:3037] T fcb6c5d1d07843388690dd522e6f18bc P 802eb57061d9457e817c3e700330b143 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:25:35.363958 8581 raft_consensus.cc:504] T fcb6c5d1d07843388690dd522e6f18bc P 802eb57061d9457e817c3e700330b143 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69302204002a49cea7f3730f16da5a25" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 36963 } } peers { permanent_uuid: "802eb57061d9457e817c3e700330b143" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 34819 } } peers { permanent_uuid: "0df99bbbc2c14b959b7f0a90656988af" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 34733 } }
I20260808 18:25:35.364285 8581 leader_election.cc:288] T fcb6c5d1d07843388690dd522e6f18bc P 802eb57061d9457e817c3e700330b143 [CANDIDATE]: Term 1 election: Requested vote from peers 69302204002a49cea7f3730f16da5a25 (127.23.72.129:36963), 0df99bbbc2c14b959b7f0a90656988af (127.23.72.131:34733)
I20260808 18:25:35.364681 8220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb6c5d1d07843388690dd522e6f18bc" candidate_uuid: "802eb57061d9457e817c3e700330b143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69302204002a49cea7f3730f16da5a25"
I20260808 18:25:35.364778 8505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb6c5d1d07843388690dd522e6f18bc" candidate_uuid: "802eb57061d9457e817c3e700330b143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0df99bbbc2c14b959b7f0a90656988af"
I20260808 18:25:35.364830 8220 raft_consensus.cc:3037] T fcb6c5d1d07843388690dd522e6f18bc P 69302204002a49cea7f3730f16da5a25 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:25:35.364960 8505 raft_consensus.cc:3037] T fcb6c5d1d07843388690dd522e6f18bc P 0df99bbbc2c14b959b7f0a90656988af [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:25:35.365898 8220 raft_consensus.cc:2454] T fcb6c5d1d07843388690dd522e6f18bc P 69302204002a49cea7f3730f16da5a25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 802eb57061d9457e817c3e700330b143 in term 1.
I20260808 18:25:35.365947 8505 raft_consensus.cc:2454] T fcb6c5d1d07843388690dd522e6f18bc P 0df99bbbc2c14b959b7f0a90656988af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 802eb57061d9457e817c3e700330b143 in term 1.
I20260808 18:25:35.366216 8292 leader_election.cc:302] T fcb6c5d1d07843388690dd522e6f18bc P 802eb57061d9457e817c3e700330b143 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0df99bbbc2c14b959b7f0a90656988af, 802eb57061d9457e817c3e700330b143; no voters:
I20260808 18:25:35.366449 8581 raft_consensus.cc:2781] T fcb6c5d1d07843388690dd522e6f18bc P 802eb57061d9457e817c3e700330b143 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:25:35.366704 8581 raft_consensus.cc:686] T fcb6c5d1d07843388690dd522e6f18bc P 802eb57061d9457e817c3e700330b143 [term 1 LEADER]: Becoming Leader. State: Replica: 802eb57061d9457e817c3e700330b143, State: Running, Role: LEADER
I20260808 18:25:35.367026 8581 consensus_queue.cc:237] T fcb6c5d1d07843388690dd522e6f18bc P 802eb57061d9457e817c3e700330b143 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69302204002a49cea7f3730f16da5a25" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 36963 } } peers { permanent_uuid: "802eb57061d9457e817c3e700330b143" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 34819 } } peers { permanent_uuid: "0df99bbbc2c14b959b7f0a90656988af" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 34733 } }
I20260808 18:25:35.370229 8065 catalog_manager.cc:5495] T fcb6c5d1d07843388690dd522e6f18bc P 802eb57061d9457e817c3e700330b143 reported cstate change: term changed from 0 to 1, leader changed from <none> to 802eb57061d9457e817c3e700330b143 (127.23.72.130). New cstate: current_term: 1 leader_uuid: "802eb57061d9457e817c3e700330b143" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69302204002a49cea7f3730f16da5a25" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 36963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "802eb57061d9457e817c3e700330b143" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 34819 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0df99bbbc2c14b959b7f0a90656988af" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 34733 } health_report { overall_health: UNKNOWN } } }
I20260808 18:25:35.396903 8570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:35.393302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41610 (local address 127.23.72.130:34819)
0808 18:25:35.393481 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:35.393487 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:35.393501 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:25:35.393572 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:35.393574 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:35.393624 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:35.393733 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:25:35.393988 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:35.394509 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:35.395377 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:35.395649 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:35.395761 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:35.396085 (+ 324us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:25:35.396477 (+ 392us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:25:35.396612 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:35.396674 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 18:25:35.396730 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260808 18:25:35.402916 8505 raft_consensus.cc:1261] T fcb6c5d1d07843388690dd522e6f18bc P 0df99bbbc2c14b959b7f0a90656988af [term 1 FOLLOWER]: Refusing update from remote peer 802eb57061d9457e817c3e700330b143: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:25:35.403105 8220 raft_consensus.cc:1261] T fcb6c5d1d07843388690dd522e6f18bc P 69302204002a49cea7f3730f16da5a25 [term 1 FOLLOWER]: Refusing update from remote peer 802eb57061d9457e817c3e700330b143: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:25:35.403718 8584 consensus_queue.cc:1035] T fcb6c5d1d07843388690dd522e6f18bc P 802eb57061d9457e817c3e700330b143 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0df99bbbc2c14b959b7f0a90656988af" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 34733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:25:35.403903 8581 consensus_queue.cc:1035] T fcb6c5d1d07843388690dd522e6f18bc P 802eb57061d9457e817c3e700330b143 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69302204002a49cea7f3730f16da5a25" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 36963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:25:35.411504 8588 mvcc.cc:205] Tried to move back new op lower bound from 7316330641002946560 to 7316330640866066432. Current Snapshot: MvccSnapshot[applied={T|T < 7316330641002946560}]
I20260808 18:25:35.413307 8589 mvcc.cc:205] Tried to move back new op lower bound from 7316330641002946560 to 7316330640866066432. Current Snapshot: MvccSnapshot[applied={T|T < 7316330641002946560}]
I20260808 18:25:35.420063 8064 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45716:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:25:35.420208 8064 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45716:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:25:35.422605 8064 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d: Sending DeleteTablet for 3 replicas of tablet fcb6c5d1d07843388690dd522e6f18bc
I20260808 18:25:35.423363 8199 tablet_service.cc:1514] Processing DeleteTablet for tablet fcb6c5d1d07843388690dd522e6f18bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:25:35 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:55684
I20260808 18:25:35.423477 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 8132
I20260808 18:25:35.424108 8485 tablet_service.cc:1514] Processing DeleteTablet for tablet fcb6c5d1d07843388690dd522e6f18bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:25:35 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:60018
I20260808 18:25:35.424551 8340 tablet_service.cc:1514] Processing DeleteTablet for tablet fcb6c5d1d07843388690dd522e6f18bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:25:35 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:41602
I20260808 18:25:35.425030 8597 tablet_replica.cc:330] T fcb6c5d1d07843388690dd522e6f18bc P 802eb57061d9457e817c3e700330b143: stopping tablet replica
I20260808 18:25:35.425422 8597 raft_consensus.cc:2229] T fcb6c5d1d07843388690dd522e6f18bc P 802eb57061d9457e817c3e700330b143 [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:25:35.425868 8597 raft_consensus.cc:2258] T fcb6c5d1d07843388690dd522e6f18bc P 802eb57061d9457e817c3e700330b143 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:25:35.427189 8597 ts_tablet_manager.cc:1905] T fcb6c5d1d07843388690dd522e6f18bc P 802eb57061d9457e817c3e700330b143: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:25:35.429473 8597 ts_tablet_manager.cc:1918] T fcb6c5d1d07843388690dd522e6f18bc P 802eb57061d9457e817c3e700330b143: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:25:35.429590 8597 log.cc:1196] T fcb6c5d1d07843388690dd522e6f18bc P 802eb57061d9457e817c3e700330b143: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-1/wal/wals/fcb6c5d1d07843388690dd522e6f18bc
I20260808 18:25:35.429819 8596 tablet_replica.cc:330] T fcb6c5d1d07843388690dd522e6f18bc P 0df99bbbc2c14b959b7f0a90656988af: stopping tablet replica
I20260808 18:25:35.430037 8597 ts_tablet_manager.cc:1939] T fcb6c5d1d07843388690dd522e6f18bc P 802eb57061d9457e817c3e700330b143: Deleting consensus metadata
I20260808 18:25:35.430289 8596 raft_consensus.cc:2229] T fcb6c5d1d07843388690dd522e6f18bc P 0df99bbbc2c14b959b7f0a90656988af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:25:35.430655 8596 raft_consensus.cc:2258] T fcb6c5d1d07843388690dd522e6f18bc P 0df99bbbc2c14b959b7f0a90656988af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:25:35.431296 8052 catalog_manager.cc:4841] TS 802eb57061d9457e817c3e700330b143 (127.23.72.130:34819): tablet fcb6c5d1d07843388690dd522e6f18bc (table test-table [id=10fc89082ea643b1adcc72bedbec4575]) successfully deleted
I20260808 18:25:35.432237 8596 ts_tablet_manager.cc:1905] T fcb6c5d1d07843388690dd522e6f18bc P 0df99bbbc2c14b959b7f0a90656988af: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 18:25:35.433997 8052 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.72.129:36963 (error 108)
I20260808 18:25:35.434345 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 8273
I20260808 18:25:35.434653 8596 ts_tablet_manager.cc:1918] T fcb6c5d1d07843388690dd522e6f18bc P 0df99bbbc2c14b959b7f0a90656988af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:25:35.434762 8596 log.cc:1196] T fcb6c5d1d07843388690dd522e6f18bc P 0df99bbbc2c14b959b7f0a90656988af: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-2/wal/wals/fcb6c5d1d07843388690dd522e6f18bc
I20260808 18:25:35.435608 8596 ts_tablet_manager.cc:1939] T fcb6c5d1d07843388690dd522e6f18bc P 0df99bbbc2c14b959b7f0a90656988af: Deleting consensus metadata
I20260808 18:25:35.436410 8569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:35.435910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:36963 (local address 127.0.0.1:55698)
0808 18:25:35.436197 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:35.436296 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.129:36963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180}
W20260808 18:25:35.436882 8052 catalog_manager.cc:4584] TS 69302204002a49cea7f3730f16da5a25 (127.23.72.129:36963): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fcb6c5d1d07843388690dd522e6f18bc: Network error: Client connection negotiation failed: client connection to 127.23.72.129:36963: connect: Connection refused (error 111)
I20260808 18:25:35.437120 8052 catalog_manager.cc:4841] TS 0df99bbbc2c14b959b7f0a90656988af (127.23.72.131:34733): tablet fcb6c5d1d07843388690dd522e6f18bc (table test-table [id=10fc89082ea643b1adcc72bedbec4575]) successfully deleted
I20260808 18:25:35.442535 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 8417
I20260808 18:25:35.449790 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 8032
I20260808 18:25:55.458592 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:46191
--webserver_interface=127.23.72.190
--webserver_port=42271
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:45461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:46191
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:25:55.574703 8599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:25:55.574961 8599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:25:55.575006 8599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:25:55.578991 8599 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 18:25:55.579056 8599 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 18:25:55.579072 8599 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 18:25:55.579087 8599 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:25:55.579104 8599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:25:55.579123 8599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:25:55.579190 8599 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:25:55.579210 8599 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:25:55.584548 8599 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:45461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:46191
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:46191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.72.190
--webserver_port=42271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8599
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:25:55.585639 8599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:25:55.586673 8599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 18:25:55.600733 8599 server_base.cc:1034] running on GCE node
W20260808 18:25:55.600687 8605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:55.600683 8608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:55.600672 8606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:25:55.601639 8599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:25:55.602828 8599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:25:55.604022 8599 hybrid_clock.cc:648] HybridClock initialized: now 1786213555604010 us; error 27 us; skew 500 ppm
Aug 08 18:25:55 dist-test-slave-5k54 krb5kdc[8019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213555, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:25:55.607360 8599 init.cc:377] Logged in from keytab as kudu/127.23.72.190@KRBTEST.COM (short username kudu)
I20260808 18:25:55.608448 8599 webserver.cc:460] Webserver started at http://127.23.72.190:42271/ using document root <none> and password file <none>
I20260808 18:25:55.609153 8599 fs_manager.cc:359] Metadata directory not provided
I20260808 18:25:55.609232 8599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:25:55.613232 8599 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 18:25:55.615376 8615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:55.616660 8599 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:25:55.616801 8599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "42c149b9c53a4e80a854d8db25ac438d"
format_stamp: "Formatted at 2026-08-08 18:25:34 on dist-test-slave-5k54"
server_key: "ffc482c5a6c05b21ee579622e101f4e0"
server_key_iv: "89e73e4faaa2d7598ff1a669c1d02846"
server_key_version: "encryptionkey@0"
I20260808 18:25:55.617476 8599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:25:55.646739 8599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:25:55.649857 8599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:25:55.650111 8599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:25:55.658195 8599 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.190:46191
I20260808 18:25:55.658494 8671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.190:46191 every 8 connection(s)
I20260808 18:25:55.659374 8599 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:25:55.663802 8672 sys_catalog.cc:263] Verifying existing consensus state
I20260808 18:25:55.665488 8672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d: Bootstrap starting.
I20260808 18:25:55.667256 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 8599
I20260808 18:25:55.667783 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.129:36963
--local_ip_for_outbound_sockets=127.23.72.129
--tserver_master_addrs=127.23.72.190:46191
--webserver_port=45303
--webserver_interface=127.23.72.129
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab
--principal=kudu/127.23.72.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:45461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:25:55.684429 8672 log.cc:824] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d: Log is configured to *not* fsync() on all Append() calls
I20260808 18:25:55.693802 8672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 18:25:55.694206 8672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d: Bootstrap complete.
I20260808 18:25:55.698357 8672 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c149b9c53a4e80a854d8db25ac438d" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 46191 } }
I20260808 18:25:55.698756 8672 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42c149b9c53a4e80a854d8db25ac438d, State: Initialized, Role: FOLLOWER
I20260808 18:25:55.699250 8672 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c149b9c53a4e80a854d8db25ac438d" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 46191 } }
I20260808 18:25:55.699368 8672 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:25:55.699483 8672 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:25:55.699611 8672 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [term 1 FOLLOWER]: Advancing to term 2
I20260808 18:25:55.701532 8672 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c149b9c53a4e80a854d8db25ac438d" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 46191 } }
I20260808 18:25:55.701879 8672 leader_election.cc:302] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42c149b9c53a4e80a854d8db25ac438d; no voters:
I20260808 18:25:55.702245 8672 leader_election.cc:288] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 18:25:55.702332 8676 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [term 2 FOLLOWER]: Leader election won for term 2
I20260808 18:25:55.702584 8676 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [term 2 LEADER]: Becoming Leader. State: Replica: 42c149b9c53a4e80a854d8db25ac438d, State: Running, Role: LEADER
I20260808 18:25:55.703027 8676 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c149b9c53a4e80a854d8db25ac438d" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 46191 } }
I20260808 18:25:55.703533 8672 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:25:55.704487 8677 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "42c149b9c53a4e80a854d8db25ac438d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c149b9c53a4e80a854d8db25ac438d" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 46191 } } }
I20260808 18:25:55.704634 8677 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [sys.catalog]: This master's current role is: LEADER
I20260808 18:25:55.705389 8685 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:25:55.705395 8678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42c149b9c53a4e80a854d8db25ac438d. Latest consensus state: current_term: 2 leader_uuid: "42c149b9c53a4e80a854d8db25ac438d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c149b9c53a4e80a854d8db25ac438d" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 46191 } } }
I20260808 18:25:55.706180 8678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [sys.catalog]: This master's current role is: LEADER
I20260808 18:25:55.710362 8685 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:25:55.710964 8685 catalog_manager.cc:1259] Loaded cluster ID: cfbae75465cc4c53a9de1e6a0f1a7efa
I20260808 18:25:55.711025 8685 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:25:55.713583 8685 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:25:55.714267 8685 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d: Loaded TSK: 0
I20260808 18:25:55.730194 8685 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d: Generated new TSK 1
I20260808 18:25:55.730454 8685 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 18:25:55.788213 8674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:25:55.788460 8674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:25:55.788504 8674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:25:55.792742 8674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:25:55.792805 8674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:25:55.792863 8674 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.129
I20260808 18:25:55.798000 8674 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:45461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab
--principal=kudu/127.23.72.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.129:36963
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.72.129
--webserver_port=45303
--tserver_master_addrs=127.23.72.190:46191
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8674
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.129
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:25:55.798986 8674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:25:55.800007 8674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:25:55.810662 8701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:55.810665 8702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:55.810694 8705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:25:55.810889 8674 server_base.cc:1034] running on GCE node
I20260808 18:25:55.811354 8674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:25:55.812041 8674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:25:55.813237 8674 hybrid_clock.cc:648] HybridClock initialized: now 1786213555813210 us; error 46 us; skew 500 ppm
Aug 08 18:25:55 dist-test-slave-5k54 krb5kdc[8019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213555, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:25:55.816254 8674 init.cc:377] Logged in from keytab as kudu/127.23.72.129@KRBTEST.COM (short username kudu)
I20260808 18:25:55.817381 8674 webserver.cc:460] Webserver started at http://127.23.72.129:45303/ using document root <none> and password file <none>
I20260808 18:25:55.818110 8674 fs_manager.cc:359] Metadata directory not provided
I20260808 18:25:55.818233 8674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:25:55.823661 8674 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 18:25:55.826411 8712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:55.828015 8674 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:25:55.828140 8674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-0/wal
uuid: "69302204002a49cea7f3730f16da5a25"
format_stamp: "Formatted at 2026-08-08 18:25:34 on dist-test-slave-5k54"
server_key: "756f5900f5382ef341510d7ca5b9494f"
server_key_iv: "7f3f90e976bcfcb36899215563a6cfe9"
server_key_version: "encryptionkey@0"
I20260808 18:25:55.828838 8674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:25:55.847709 8674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:25:55.851696 8674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:25:55.851897 8674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:25:55.852592 8674 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:25:55.853874 8719 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 18:25:55.858203 8674 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 18:25:55.858297 8674 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260808 18:25:55.858361 8674 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 18:25:55.860509 8674 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 18:25:55.860570 8674 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260808 18:25:55.860960 8719 tablet_bootstrap.cc:492] T fcb6c5d1d07843388690dd522e6f18bc P 69302204002a49cea7f3730f16da5a25: Bootstrap starting.
I20260808 18:25:55.871883 8674 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.129:36963
I20260808 18:25:55.871946 8834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.129:36963 every 8 connection(s)
I20260808 18:25:55.872884 8674 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
I20260808 18:25:55.875717 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 8674
I20260808 18:25:55.876228 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.130:34819
--local_ip_for_outbound_sockets=127.23.72.130
--tserver_master_addrs=127.23.72.190:46191
--webserver_port=38803
--webserver_interface=127.23.72.130
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab
--principal=kudu/127.23.72.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:45461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:25:55.882081 8719 log.cc:824] T fcb6c5d1d07843388690dd522e6f18bc P 69302204002a49cea7f3730f16da5a25: Log is configured to *not* fsync() on all Append() calls
Aug 08 18:25:55 dist-test-slave-5k54 krb5kdc[8019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213555, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:25:55.890761 8719 tablet_bootstrap.cc:492] T fcb6c5d1d07843388690dd522e6f18bc P 69302204002a49cea7f3730f16da5a25: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 18:25:55.891249 8719 tablet_bootstrap.cc:492] T fcb6c5d1d07843388690dd522e6f18bc P 69302204002a49cea7f3730f16da5a25: Bootstrap complete.
I20260808 18:25:55.892719 8719 ts_tablet_manager.cc:1397] T fcb6c5d1d07843388690dd522e6f18bc P 69302204002a49cea7f3730f16da5a25: Time spent bootstrapping tablet: real 0.032s user 0.026s sys 0.004s
I20260808 18:25:55.894368 8837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:55.875356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:48701 (local address 127.23.72.190:46191)
0808 18:25:55.875923 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:55.875944 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:55.876525 (+ 581us) server_negotiation.cc:407] Connection header received
0808 18:25:55.878045 (+ 1520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:55.878065 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:55.878516 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:55.878906 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:25:55.880552 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:55.881598 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:55.883030 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:55.883916 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:55.884147 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:55.887723 (+ 3576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:25:55.887730 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:25:55.887751 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:25:55.887787 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:25:55.890172 (+ 2385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:55.891023 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:55.891033 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:55.891041 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:55.891212 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:55.891755 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:55.891762 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:55.891767 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:55.892304 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:25:55.892451 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:55.893216 (+ 765us) server_negotiation.cc:299] Negotiation successful
0808 18:25:55.893528 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":150,"threads_started":1}
I20260808 18:25:55.894414 8838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:55.875311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.129:48701)
0808 18:25:55.876282 (+ 971us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:55.876351 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:55.877519 (+ 1168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:55.879005 (+ 1486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:55.879015 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:55.879680 (+ 665us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:25:55.880352 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:55.880374 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:55.881782 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:55.881785 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:55.882874 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:55.882888 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:55.884092 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:55.884096 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:55.884284 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:55.885048 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:25:55.885069 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:25:55.887474 (+ 2405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:25:55.890320 (+ 2846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:55.890327 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:55.890344 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:55.890826 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:55.891348 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:55.891353 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:55.891357 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:55.891618 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:55.892444 (+ 826us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:25:55.892451 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:25:55.892779 (+ 328us) client_negotiation.cc:769] Sending connection context
0808 18:25:55.893052 (+ 273us) client_negotiation.cc:240] Negotiation successful
0808 18:25:55.893415 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":725,"thread_start_us":170,"threads_started":1}
I20260808 18:25:55.895773 8719 raft_consensus.cc:348] T fcb6c5d1d07843388690dd522e6f18bc P 69302204002a49cea7f3730f16da5a25 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69302204002a49cea7f3730f16da5a25" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 36963 } } peers { permanent_uuid: "802eb57061d9457e817c3e700330b143" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 34819 } } peers { permanent_uuid: "0df99bbbc2c14b959b7f0a90656988af" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 34733 } }
I20260808 18:25:55.896199 8719 raft_consensus.cc:729] T fcb6c5d1d07843388690dd522e6f18bc P 69302204002a49cea7f3730f16da5a25 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69302204002a49cea7f3730f16da5a25, State: Initialized, Role: FOLLOWER
I20260808 18:25:55.896698 8719 consensus_queue.cc:260] T fcb6c5d1d07843388690dd522e6f18bc P 69302204002a49cea7f3730f16da5a25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69302204002a49cea7f3730f16da5a25" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 36963 } } peers { permanent_uuid: "802eb57061d9457e817c3e700330b143" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 34819 } } peers { permanent_uuid: "0df99bbbc2c14b959b7f0a90656988af" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 34733 } }
I20260808 18:25:55.897707 8719 ts_tablet_manager.cc:1428] T fcb6c5d1d07843388690dd522e6f18bc P 69302204002a49cea7f3730f16da5a25: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 18:25:55.898011 8835 heartbeater.cc:347] Connected to a master server at 127.23.72.190:46191
I20260808 18:25:55.898262 8835 heartbeater.cc:464] Registering TS with master...
I20260808 18:25:55.898834 8835 heartbeater.cc:511] Master 127.23.72.190:46191 requested a full tablet report, sending...
I20260808 18:25:55.901425 8633 ts_manager.cc:194] Registered new tserver with Master: 69302204002a49cea7f3730f16da5a25 (127.23.72.129:36963)
I20260808 18:25:55.904975 8633 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.129@KRBTEST.COM'} at 127.23.72.129:48701
I20260808 18:25:55.907593 8835 heartbeater.cc:503] Master 127.23.72.190:46191 was elected leader, sending a full tablet report...
Aug 08 18:25:55 dist-test-slave-5k54 krb5kdc[8019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213555, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for kudu/127.23.72.129@KRBTEST.COM
I20260808 18:25:55.916613 8843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:55.904440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50514 (local address 127.23.72.129:36963)
0808 18:25:55.905833 (+ 1393us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:55.905848 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:55.905873 (+ 25us) server_negotiation.cc:407] Connection header received
0808 18:25:55.905943 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:55.905948 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:55.906159 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:55.906308 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:25:55.907041 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:55.908264 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:55.909484 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:55.910018 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:55.910138 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:55.912751 (+ 2613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:25:55.912760 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:25:55.912770 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:25:55.912791 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:25:55.914672 (+ 1881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:55.915264 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:55.915268 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:55.915270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:55.915345 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:55.915676 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:55.915679 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:55.915680 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:55.915884 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:25:55.915990 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:55.916362 (+ 372us) server_negotiation.cc:299] Negotiation successful
0808 18:25:55.916491 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1277,"thread_start_us":131,"threads_started":1}
I20260808 18:25:55.916675 8842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:55.904370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:36963 (local address 127.0.0.1:50514)
0808 18:25:55.905329 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:55.905380 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:55.905531 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:55.906332 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:55.906338 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:55.906782 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:25:55.906907 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:55.906915 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:55.908394 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:55.908397 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:55.909365 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:55.909374 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:55.910173 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:55.910176 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:55.910262 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:55.910859 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:25:55.910871 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:25:55.912611 (+ 1740us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:25:55.914801 (+ 2190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:55.914811 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:55.914816 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:55.915143 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:55.915466 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:55.915470 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:55.915472 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:55.915579 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:55.916000 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:25:55.916006 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:25:55.916136 (+ 130us) client_negotiation.cc:769] Sending connection context
0808 18:25:55.916294 (+ 158us) client_negotiation.cc:240] Negotiation successful
0808 18:25:55.916500 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":754,"thread_start_us":115,"threads_started":1}
I20260808 18:25:55.917652 8763 tablet_service.cc:1514] Processing DeleteTablet for tablet fcb6c5d1d07843388690dd522e6f18bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:25:35 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:50514
I20260808 18:25:55.918702 8844 tablet_replica.cc:330] stopping tablet replica
I20260808 18:25:55.919010 8844 raft_consensus.cc:2229] T fcb6c5d1d07843388690dd522e6f18bc P 69302204002a49cea7f3730f16da5a25 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:25:55.919253 8844 raft_consensus.cc:2258] T fcb6c5d1d07843388690dd522e6f18bc P 69302204002a49cea7f3730f16da5a25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:25:55.920202 8844 ts_tablet_manager.cc:1905] T fcb6c5d1d07843388690dd522e6f18bc P 69302204002a49cea7f3730f16da5a25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:25:55.923895 8844 ts_tablet_manager.cc:1918] T fcb6c5d1d07843388690dd522e6f18bc P 69302204002a49cea7f3730f16da5a25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:25:55.923986 8844 log.cc:1196] T fcb6c5d1d07843388690dd522e6f18bc P 69302204002a49cea7f3730f16da5a25: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-0/wal/wals/fcb6c5d1d07843388690dd522e6f18bc
I20260808 18:25:55.924255 8844 ts_tablet_manager.cc:1939] T fcb6c5d1d07843388690dd522e6f18bc P 69302204002a49cea7f3730f16da5a25: Deleting consensus metadata
I20260808 18:25:55.925907 8619 catalog_manager.cc:4841] TS 69302204002a49cea7f3730f16da5a25 (127.23.72.129:36963): tablet fcb6c5d1d07843388690dd522e6f18bc (table test-table [id=10fc89082ea643b1adcc72bedbec4575]) successfully deleted
W20260808 18:25:56.006560 8839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:25:56.006830 8839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:25:56.006913 8839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:25:56.011060 8839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:25:56.011150 8839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:25:56.011245 8839 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.130
I20260808 18:25:56.016526 8839 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:45461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab
--principal=kudu/127.23.72.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.130:34819
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.72.130
--webserver_port=38803
--tserver_master_addrs=127.23.72.190:46191
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8839
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.130
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:25:56.017640 8839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:25:56.018651 8839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:25:56.028514 8849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:56.028539 8850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:56.028520 8853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:25:56.028784 8839 server_base.cc:1034] running on GCE node
I20260808 18:25:56.029234 8839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:25:56.029901 8839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:25:56.031124 8839 hybrid_clock.cc:648] HybridClock initialized: now 1786213556031111 us; error 27 us; skew 500 ppm
Aug 08 18:25:56 dist-test-slave-5k54 krb5kdc[8019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213556, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:25:56.034025 8839 init.cc:377] Logged in from keytab as kudu/127.23.72.130@KRBTEST.COM (short username kudu)
I20260808 18:25:56.035018 8839 webserver.cc:460] Webserver started at http://127.23.72.130:38803/ using document root <none> and password file <none>
I20260808 18:25:56.035702 8839 fs_manager.cc:359] Metadata directory not provided
I20260808 18:25:56.035784 8839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:25:56.039608 8839 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:25:56.041679 8860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:56.042907 8839 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:25:56.043028 8839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-1/wal
uuid: "802eb57061d9457e817c3e700330b143"
format_stamp: "Formatted at 2026-08-08 18:25:34 on dist-test-slave-5k54"
server_key: "27a71935d215f18387c4d896c480cbbc"
server_key_iv: "715815ef6fedf61ff01271c330ae925d"
server_key_version: "encryptionkey@0"
I20260808 18:25:56.043637 8839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:25:56.070895 8839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:25:56.074029 8839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:25:56.074240 8839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:25:56.074884 8839 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:25:56.075805 8839 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:25:56.075857 8839 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:56.075898 8839 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:25:56.075956 8839 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:56.085582 8839 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.130:34819
I20260808 18:25:56.085601 8982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.130:34819 every 8 connection(s)
I20260808 18:25:56.086632 8839 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
I20260808 18:25:56.096273 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 8839
Aug 08 18:25:56 dist-test-slave-5k54 krb5kdc[8019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213556, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.130@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:25:56.096850 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.131:34733
--local_ip_for_outbound_sockets=127.23.72.131
--tserver_master_addrs=127.23.72.190:46191
--webserver_port=33835
--webserver_interface=127.23.72.131
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab
--principal=kudu/127.23.72.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:45461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:25:56.101140 8837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:56.088494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:44065 (local address 127.23.72.190:46191)
0808 18:25:56.088642 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:56.088646 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:56.089543 (+ 897us) server_negotiation.cc:407] Connection header received
0808 18:25:56.091012 (+ 1469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:56.091016 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:56.091078 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:56.091184 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:25:56.092457 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:56.093001 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:56.094137 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:56.094379 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:56.094498 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:56.097252 (+ 2754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:25:56.097255 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:25:56.097257 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:25:56.097281 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:25:56.098832 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:56.099346 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:56.099353 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:56.099357 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:56.099493 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:56.099829 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:56.099835 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:56.099840 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:56.100121 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:25:56.100232 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:56.100707 (+ 475us) server_negotiation.cc:299] Negotiation successful
0808 18:25:56.100874 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 18:25:56.101753 8985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:56.088825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.130:44065)
0808 18:25:56.089373 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:56.089420 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:56.090776 (+ 1356us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:56.091383 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:56.091389 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:56.091818 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:25:56.092294 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:56.092307 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:56.093132 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:56.093135 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:56.094002 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:56.094011 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:56.094554 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:56.094559 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:56.094720 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:56.095344 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:25:56.095360 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:25:56.097071 (+ 1711us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:25:56.098968 (+ 1897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:56.098973 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:56.098986 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:56.099243 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:56.099594 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:56.099597 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:56.099598 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:56.099731 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:56.100229 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:25:56.100234 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:25:56.100460 (+ 226us) client_negotiation.cc:769] Sending connection context
0808 18:25:56.100615 (+ 155us) client_negotiation.cc:240] Negotiation successful
0808 18:25:56.100874 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":126,"threads_started":1}
I20260808 18:25:56.103013 8983 heartbeater.cc:347] Connected to a master server at 127.23.72.190:46191
I20260808 18:25:56.103286 8983 heartbeater.cc:464] Registering TS with master...
I20260808 18:25:56.103915 8983 heartbeater.cc:511] Master 127.23.72.190:46191 requested a full tablet report, sending...
I20260808 18:25:56.105125 8633 ts_manager.cc:194] Registered new tserver with Master: 802eb57061d9457e817c3e700330b143 (127.23.72.130:34819)
I20260808 18:25:56.105822 8633 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.130@KRBTEST.COM'} at 127.23.72.130:44065
W20260808 18:25:56.211661 8986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:25:56.211896 8986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:25:56.211966 8986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:25:56.215968 8986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:25:56.216027 8986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:25:56.216120 8986 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.131
I20260808 18:25:56.221549 8986 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:45461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab
--principal=kudu/127.23.72.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.131:34733
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.72.131
--webserver_port=33835
--tserver_master_addrs=127.23.72.190:46191
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8986
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.131
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:25:56.222631 8986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:25:56.223739 8986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:25:56.233408 8993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:56.233404 8992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:25:56.233516 8995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:25:56.233644 8986 server_base.cc:1034] running on GCE node
I20260808 18:25:56.234194 8986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:25:56.234858 8986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:25:56.236043 8986 hybrid_clock.cc:648] HybridClock initialized: now 1786213556236018 us; error 49 us; skew 500 ppm
Aug 08 18:25:56 dist-test-slave-5k54 krb5kdc[8019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213556, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:25:56.238816 8986 init.cc:377] Logged in from keytab as kudu/127.23.72.131@KRBTEST.COM (short username kudu)
I20260808 18:25:56.239715 8986 webserver.cc:460] Webserver started at http://127.23.72.131:33835/ using document root <none> and password file <none>
I20260808 18:25:56.240269 8986 fs_manager.cc:359] Metadata directory not provided
I20260808 18:25:56.240377 8986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:25:56.244125 8986 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 18:25:56.246131 9002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:56.247398 8986 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:25:56.247606 8986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-2/wal
uuid: "0df99bbbc2c14b959b7f0a90656988af"
format_stamp: "Formatted at 2026-08-08 18:25:35 on dist-test-slave-5k54"
server_key: "66d1bf5677d875d05f11dce131dc72bc"
server_key_iv: "8428f367a03d10657fdf1b3da9a8eb69"
server_key_version: "encryptionkey@0"
I20260808 18:25:56.248190 8986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:25:56.268280 8986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:25:56.271703 8986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:25:56.271907 8986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:25:56.272575 8986 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:25:56.273406 8986 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:25:56.273485 8986 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:56.273561 8986 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:25:56.273612 8986 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:25:56.282528 8986 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.131:34733
I20260808 18:25:56.282562 9124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.131:34733 every 8 connection(s)
I20260808 18:25:56.283577 8986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
I20260808 18:25:56.285053 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 8986
Aug 08 18:25:56 dist-test-slave-5k54 krb5kdc[8019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213556, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.131@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:25:56.294540 9127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:56.285744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39992 (local address 127.23.72.190:46191)
0808 18:25:56.285972 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:56.285977 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:56.286164 (+ 187us) server_negotiation.cc:407] Connection header received
0808 18:25:56.286343 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:56.286346 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:56.286403 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:56.286505 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:25:56.287284 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:56.287924 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:56.288943 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:56.289292 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:56.289405 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:56.290969 (+ 1564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:25:56.290973 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:25:56.290975 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:25:56.291002 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:25:56.292816 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:56.293260 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:56.293265 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:56.293267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:56.293357 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:56.293651 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:56.293654 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:56.293655 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:56.293833 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:25:56.294054 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:56.294201 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 18:25:56.294321 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":95,"threads_started":1}
I20260808 18:25:56.297904 8837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:56.285437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:47701 (local address 127.23.72.190:46191)
0808 18:25:56.285641 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:56.285646 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:56.286573 (+ 927us) server_negotiation.cc:407] Connection header received
0808 18:25:56.288042 (+ 1469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:56.288047 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:56.288101 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:56.288178 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:25:56.289525 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:56.290251 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:56.291756 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:56.291973 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:56.292110 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:56.294457 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:25:56.294461 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:25:56.294463 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:25:56.294492 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:25:56.295947 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:56.296445 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:56.296449 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:56.296451 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:56.296531 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:56.296864 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:56.296869 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:56.296871 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:56.297036 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:25:56.297142 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:56.297621 (+ 479us) server_negotiation.cc:299] Negotiation successful
0808 18:25:56.297733 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260808 18:25:56.298678 9129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:56.285931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.131:47701)
0808 18:25:56.286425 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:56.286477 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:56.287843 (+ 1366us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:56.288324 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:56.288330 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:56.288670 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:25:56.289373 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:56.289394 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:56.290396 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:56.290399 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:56.291609 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:56.291621 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:56.292147 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:56.292151 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:56.292335 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:56.292884 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:25:56.292900 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:25:56.294299 (+ 1399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:25:56.296060 (+ 1761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:56.296066 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:56.296079 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:56.296340 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:56.296623 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:25:56.296627 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:25:56.296629 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:25:56.296766 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:25:56.297142 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:25:56.297150 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:25:56.297395 (+ 245us) client_negotiation.cc:769] Sending connection context
0808 18:25:56.297577 (+ 182us) client_negotiation.cc:240] Negotiation successful
0808 18:25:56.297882 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":140,"threads_started":1}
I20260808 18:25:56.299847 9125 heartbeater.cc:347] Connected to a master server at 127.23.72.190:46191
I20260808 18:25:56.300061 9125 heartbeater.cc:464] Registering TS with master...
I20260808 18:25:56.300486 9125 heartbeater.cc:511] Master 127.23.72.190:46191 requested a full tablet report, sending...
I20260808 18:25:56.301508 8633 ts_manager.cc:194] Registered new tserver with Master: 0df99bbbc2c14b959b7f0a90656988af (127.23.72.131:34733)
I20260808 18:25:56.302075 8633 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.131@KRBTEST.COM'} at 127.23.72.131:47701
I20260808 18:25:56.308281 23842 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:25:56.311971 8837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:56.308860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40004 (local address 127.23.72.190:46191)
0808 18:25:56.309004 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:56.309007 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:56.309185 (+ 178us) server_negotiation.cc:407] Connection header received
0808 18:25:56.309297 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:56.309298 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:56.309384 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:56.309466 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:25:56.309683 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:56.310172 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:56.310976 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:56.311177 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:56.311249 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:56.311556 (+ 307us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:25:56.311671 (+ 115us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0808 18:25:56.311861 (+ 190us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40004: token expired
Metrics: {"server-negotiator.queue_time_us":48}
W20260808 18:25:56.312053 8837 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40004: token expired
W20260808 18:25:56.312392 8559 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.23.72.190:46191)
I20260808 18:25:56.318905 8837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:56.312744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40012 (local address 127.23.72.190:46191)
0808 18:25:56.312851 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:56.312855 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:56.312877 (+ 22us) server_negotiation.cc:407] Connection header received
0808 18:25:56.312965 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:56.312967 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:56.312998 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:56.313057 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:25:56.313635 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:56.314092 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:56.314820 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:56.315027 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:56.315097 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:56.316343 (+ 1246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:25:56.316346 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:25:56.316348 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:25:56.316369 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:25:56.317605 (+ 1236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:56.317995 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:56.317997 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:56.317998 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:56.318070 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:25:56.318293 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:25:56.318295 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:25:56.318296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:25:56.318440 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:25:56.318537 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:56.318721 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 18:25:56.318806 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 18:25:56.321280 8633 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40012:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:25:56.324518 8633 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:25:56.332341 8763 tablet_service.cc:1467] Processing CreateTablet for tablet e3174a6300974b36bc1811929401b8fa (DEFAULT_TABLE table=test-table [id=1fd1c4033e304ec19f59b09a478426a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:25:56.333037 8763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3174a6300974b36bc1811929401b8fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:25:56.336097 8719 tablet_bootstrap.cc:492] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25: Bootstrap starting.
I20260808 18:25:56.336613 8842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:56.331067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.131:34733 (local address 127.0.0.1:42194)
0808 18:25:56.331172 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:56.331190 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:56.331269 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:56.332539 (+ 1270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:56.332543 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:56.332563 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:25:56.332652 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:56.332658 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:56.334251 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:56.334255 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:56.335330 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:56.335339 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:56.336206 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:56.336209 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:56.336295 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:56.336311 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 18:25:56.336362 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 18:25:56.336453 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 18:25:56.336705 9136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:56.331608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53788 (local address 127.23.72.130:34819)
0808 18:25:56.332028 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:56.332039 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:56.332059 (+ 20us) server_negotiation.cc:407] Connection header received
0808 18:25:56.332127 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:56.332131 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:56.332292 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:56.332467 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:25:56.332670 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:56.333511 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:56.335029 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:56.335916 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:56.336035 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:56.336210 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:56.336441 (+ 231us) server_negotiation.cc:299] Negotiation successful
0808 18:25:56.336552 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":93,"threads_started":1}
I20260808 18:25:56.336613 9134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:56.331315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.130:34819 (local address 127.0.0.1:53788)
0808 18:25:56.331662 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:56.331678 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:56.331767 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:56.332436 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:56.332439 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:56.332457 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:25:56.332573 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:56.332579 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:56.333675 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:56.333678 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:56.334882 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:56.334893 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:56.336053 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:56.336057 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:56.336257 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:56.336307 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 18:25:56.336389 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 18:25:56.336453 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":148,"threads_started":1}
I20260808 18:25:56.337052 8719 tablet_bootstrap.cc:654] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25: Neither blocks nor log segments found. Creating new log.
I20260808 18:25:56.337523 9135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:56.331277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42194 (local address 127.23.72.131:34733)
0808 18:25:56.332042 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:56.332055 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:56.332078 (+ 23us) server_negotiation.cc:407] Connection header received
0808 18:25:56.332157 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:56.332162 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:56.332350 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:56.332809 (+ 459us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:25:56.332976 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:56.334135 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:56.335475 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:56.336072 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:56.336911 (+ 839us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:56.337071 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:56.337122 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 18:25:56.337368 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":664,"thread_start_us":202,"threads_started":1}
I20260808 18:25:56.338642 8719 tablet_bootstrap.cc:492] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25: No bootstrap required, opened a new log
I20260808 18:25:56.338729 8719 ts_tablet_manager.cc:1397] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 18:25:56.339166 8719 raft_consensus.cc:348] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df99bbbc2c14b959b7f0a90656988af" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 34733 } } peers { permanent_uuid: "802eb57061d9457e817c3e700330b143" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 34819 } } peers { permanent_uuid: "69302204002a49cea7f3730f16da5a25" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 36963 } }
I20260808 18:25:56.339277 8719 raft_consensus.cc:374] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:25:56.339311 8719 raft_consensus.cc:729] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69302204002a49cea7f3730f16da5a25, State: Initialized, Role: FOLLOWER
I20260808 18:25:56.339668 8719 consensus_queue.cc:260] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df99bbbc2c14b959b7f0a90656988af" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 34733 } } peers { permanent_uuid: "802eb57061d9457e817c3e700330b143" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 34819 } } peers { permanent_uuid: "69302204002a49cea7f3730f16da5a25" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 36963 } }
I20260808 18:25:56.340198 8719 ts_tablet_manager.cc:1428] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:25:56.340723 9056 tablet_service.cc:1467] Processing CreateTablet for tablet e3174a6300974b36bc1811929401b8fa (DEFAULT_TABLE table=test-table [id=1fd1c4033e304ec19f59b09a478426a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:25:56.340746 8914 tablet_service.cc:1467] Processing CreateTablet for tablet e3174a6300974b36bc1811929401b8fa (DEFAULT_TABLE table=test-table [id=1fd1c4033e304ec19f59b09a478426a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:25:56.341796 9056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3174a6300974b36bc1811929401b8fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:25:56.341861 8914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3174a6300974b36bc1811929401b8fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:25:56.348546 9138 tablet_bootstrap.cc:492] T e3174a6300974b36bc1811929401b8fa P 0df99bbbc2c14b959b7f0a90656988af: Bootstrap starting.
I20260808 18:25:56.350461 9138 tablet_bootstrap.cc:654] T e3174a6300974b36bc1811929401b8fa P 0df99bbbc2c14b959b7f0a90656988af: Neither blocks nor log segments found. Creating new log.
I20260808 18:25:56.350910 9139 tablet_bootstrap.cc:492] T e3174a6300974b36bc1811929401b8fa P 802eb57061d9457e817c3e700330b143: Bootstrap starting.
I20260808 18:25:56.351858 9138 log.cc:824] T e3174a6300974b36bc1811929401b8fa P 0df99bbbc2c14b959b7f0a90656988af: Log is configured to *not* fsync() on all Append() calls
I20260808 18:25:56.353425 9139 tablet_bootstrap.cc:654] T e3174a6300974b36bc1811929401b8fa P 802eb57061d9457e817c3e700330b143: Neither blocks nor log segments found. Creating new log.
I20260808 18:25:56.353736 9138 tablet_bootstrap.cc:492] T e3174a6300974b36bc1811929401b8fa P 0df99bbbc2c14b959b7f0a90656988af: No bootstrap required, opened a new log
I20260808 18:25:56.353931 9138 ts_tablet_manager.cc:1397] T e3174a6300974b36bc1811929401b8fa P 0df99bbbc2c14b959b7f0a90656988af: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 18:25:56.354214 9139 log.cc:824] T e3174a6300974b36bc1811929401b8fa P 802eb57061d9457e817c3e700330b143: Log is configured to *not* fsync() on all Append() calls
I20260808 18:25:56.356035 9139 tablet_bootstrap.cc:492] T e3174a6300974b36bc1811929401b8fa P 802eb57061d9457e817c3e700330b143: No bootstrap required, opened a new log
I20260808 18:25:56.356238 9139 ts_tablet_manager.cc:1397] T e3174a6300974b36bc1811929401b8fa P 802eb57061d9457e817c3e700330b143: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 18:25:56.357144 9138 raft_consensus.cc:348] T e3174a6300974b36bc1811929401b8fa P 0df99bbbc2c14b959b7f0a90656988af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df99bbbc2c14b959b7f0a90656988af" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 34733 } } peers { permanent_uuid: "802eb57061d9457e817c3e700330b143" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 34819 } } peers { permanent_uuid: "69302204002a49cea7f3730f16da5a25" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 36963 } }
I20260808 18:25:56.357393 9138 raft_consensus.cc:374] T e3174a6300974b36bc1811929401b8fa P 0df99bbbc2c14b959b7f0a90656988af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:25:56.357470 9138 raft_consensus.cc:729] T e3174a6300974b36bc1811929401b8fa P 0df99bbbc2c14b959b7f0a90656988af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0df99bbbc2c14b959b7f0a90656988af, State: Initialized, Role: FOLLOWER
I20260808 18:25:56.357901 9138 consensus_queue.cc:260] T e3174a6300974b36bc1811929401b8fa P 0df99bbbc2c14b959b7f0a90656988af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df99bbbc2c14b959b7f0a90656988af" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 34733 } } peers { permanent_uuid: "802eb57061d9457e817c3e700330b143" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 34819 } } peers { permanent_uuid: "69302204002a49cea7f3730f16da5a25" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 36963 } }
I20260808 18:25:56.358528 9125 heartbeater.cc:503] Master 127.23.72.190:46191 was elected leader, sending a full tablet report...
I20260808 18:25:56.359149 9139 raft_consensus.cc:348] T e3174a6300974b36bc1811929401b8fa P 802eb57061d9457e817c3e700330b143 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df99bbbc2c14b959b7f0a90656988af" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 34733 } } peers { permanent_uuid: "802eb57061d9457e817c3e700330b143" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 34819 } } peers { permanent_uuid: "69302204002a49cea7f3730f16da5a25" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 36963 } }
I20260808 18:25:56.359380 9139 raft_consensus.cc:374] T e3174a6300974b36bc1811929401b8fa P 802eb57061d9457e817c3e700330b143 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:25:56.359463 9139 raft_consensus.cc:729] T e3174a6300974b36bc1811929401b8fa P 802eb57061d9457e817c3e700330b143 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 802eb57061d9457e817c3e700330b143, State: Initialized, Role: FOLLOWER
I20260808 18:25:56.359856 9138 ts_tablet_manager.cc:1428] T e3174a6300974b36bc1811929401b8fa P 0df99bbbc2c14b959b7f0a90656988af: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 18:25:56.359929 9139 consensus_queue.cc:260] T e3174a6300974b36bc1811929401b8fa P 802eb57061d9457e817c3e700330b143 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df99bbbc2c14b959b7f0a90656988af" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 34733 } } peers { permanent_uuid: "802eb57061d9457e817c3e700330b143" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 34819 } } peers { permanent_uuid: "69302204002a49cea7f3730f16da5a25" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 36963 } }
I20260808 18:25:56.360630 8983 heartbeater.cc:503] Master 127.23.72.190:46191 was elected leader, sending a full tablet report...
I20260808 18:25:56.360857 9139 ts_tablet_manager.cc:1428] T e3174a6300974b36bc1811929401b8fa P 802eb57061d9457e817c3e700330b143: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260808 18:25:56.374155 8836 tablet.cc:2369] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:25:56.446314 9142 raft_consensus.cc:482] T e3174a6300974b36bc1811929401b8fa P 0df99bbbc2c14b959b7f0a90656988af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:25:56.446566 9142 raft_consensus.cc:504] T e3174a6300974b36bc1811929401b8fa P 0df99bbbc2c14b959b7f0a90656988af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df99bbbc2c14b959b7f0a90656988af" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 34733 } } peers { permanent_uuid: "802eb57061d9457e817c3e700330b143" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 34819 } } peers { permanent_uuid: "69302204002a49cea7f3730f16da5a25" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 36963 } }
I20260808 18:25:56.448014 9142 leader_election.cc:288] T e3174a6300974b36bc1811929401b8fa P 0df99bbbc2c14b959b7f0a90656988af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 802eb57061d9457e817c3e700330b143 (127.23.72.130:34819), 69302204002a49cea7f3730f16da5a25 (127.23.72.129:36963)
I20260808 18:25:56.452282 9129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:56.448271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.130:34819 (local address 127.23.72.131:51183)
0808 18:25:56.448433 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:56.448453 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:56.448594 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:56.448920 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:56.448924 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:56.448948 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:25:56.449107 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:56.449114 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:56.450000 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:56.450004 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:56.451023 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:56.451032 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:56.451832 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:56.451836 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:56.451949 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:56.451976 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 18:25:56.452063 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 18:25:56.452142 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 18:25:56.452291 9136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:56.448375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:51183 (local address 127.23.72.130:34819)
0808 18:25:56.448536 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:56.448541 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:56.448560 (+ 19us) server_negotiation.cc:407] Connection header received
0808 18:25:56.448706 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:56.448709 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:56.448796 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:56.448874 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:25:56.449248 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:56.449890 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:56.451139 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:56.451725 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:56.451815 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:56.451856 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:56.452100 (+ 244us) server_negotiation.cc:299] Negotiation successful
0808 18:25:56.452174 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 18:25:56.452853 9144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:56.448567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:36963 (local address 127.23.72.131:41031)
0808 18:25:56.448932 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:56.448946 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:25:56.449046 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:25:56.449366 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:25:56.449369 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:25:56.449387 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:25:56.449505 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:56.449512 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:56.450420 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:56.450422 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:56.451369 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:25:56.451379 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:25:56.452366 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:25:56.452372 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:25:56.452522 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:56.452556 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 18:25:56.452640 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 18:25:56.452707 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":130,"threads_started":1}
I20260808 18:25:56.453058 8936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3174a6300974b36bc1811929401b8fa" candidate_uuid: "0df99bbbc2c14b959b7f0a90656988af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "802eb57061d9457e817c3e700330b143" is_pre_election: true
I20260808 18:25:56.453403 8936 raft_consensus.cc:2454] T e3174a6300974b36bc1811929401b8fa P 802eb57061d9457e817c3e700330b143 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0df99bbbc2c14b959b7f0a90656988af in term 0.
I20260808 18:25:56.453516 9147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:56.448714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:41031 (local address 127.23.72.129:36963)
0808 18:25:56.448974 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:56.448978 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:56.448991 (+ 13us) server_negotiation.cc:407] Connection header received
0808 18:25:56.449158 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:56.449164 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:56.449240 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:56.449334 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:25:56.449621 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:56.450302 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:56.451514 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:56.452222 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:56.452352 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:56.452530 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:56.452692 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 18:25:56.453374 (+ 682us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":74,"threads_started":1}
I20260808 18:25:56.453917 9006 leader_election.cc:302] T e3174a6300974b36bc1811929401b8fa P 0df99bbbc2c14b959b7f0a90656988af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0df99bbbc2c14b959b7f0a90656988af, 802eb57061d9457e817c3e700330b143; no voters:
I20260808 18:25:56.453977 8783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3174a6300974b36bc1811929401b8fa" candidate_uuid: "0df99bbbc2c14b959b7f0a90656988af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69302204002a49cea7f3730f16da5a25" is_pre_election: true
I20260808 18:25:56.454200 9142 raft_consensus.cc:2781] T e3174a6300974b36bc1811929401b8fa P 0df99bbbc2c14b959b7f0a90656988af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:25:56.454296 9142 raft_consensus.cc:482] T e3174a6300974b36bc1811929401b8fa P 0df99bbbc2c14b959b7f0a90656988af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:25:56.454345 9142 raft_consensus.cc:3037] T e3174a6300974b36bc1811929401b8fa P 0df99bbbc2c14b959b7f0a90656988af [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:25:56.454321 8783 raft_consensus.cc:2454] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0df99bbbc2c14b959b7f0a90656988af in term 0.
I20260808 18:25:56.455513 9142 raft_consensus.cc:504] T e3174a6300974b36bc1811929401b8fa P 0df99bbbc2c14b959b7f0a90656988af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df99bbbc2c14b959b7f0a90656988af" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 34733 } } peers { permanent_uuid: "802eb57061d9457e817c3e700330b143" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 34819 } } peers { permanent_uuid: "69302204002a49cea7f3730f16da5a25" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 36963 } }
I20260808 18:25:56.455922 9142 leader_election.cc:288] T e3174a6300974b36bc1811929401b8fa P 0df99bbbc2c14b959b7f0a90656988af [CANDIDATE]: Term 1 election: Requested vote from peers 802eb57061d9457e817c3e700330b143 (127.23.72.130:34819), 69302204002a49cea7f3730f16da5a25 (127.23.72.129:36963)
I20260808 18:25:56.456354 8936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3174a6300974b36bc1811929401b8fa" candidate_uuid: "0df99bbbc2c14b959b7f0a90656988af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "802eb57061d9457e817c3e700330b143"
I20260808 18:25:56.456492 8936 raft_consensus.cc:3037] T e3174a6300974b36bc1811929401b8fa P 802eb57061d9457e817c3e700330b143 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:25:56.456763 8783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3174a6300974b36bc1811929401b8fa" candidate_uuid: "0df99bbbc2c14b959b7f0a90656988af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69302204002a49cea7f3730f16da5a25"
I20260808 18:25:56.456894 8783 raft_consensus.cc:3037] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:25:56.457857 8936 raft_consensus.cc:2454] T e3174a6300974b36bc1811929401b8fa P 802eb57061d9457e817c3e700330b143 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0df99bbbc2c14b959b7f0a90656988af in term 1.
I20260808 18:25:56.458160 8783 raft_consensus.cc:2454] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0df99bbbc2c14b959b7f0a90656988af in term 1.
I20260808 18:25:56.458173 9006 leader_election.cc:302] T e3174a6300974b36bc1811929401b8fa P 0df99bbbc2c14b959b7f0a90656988af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0df99bbbc2c14b959b7f0a90656988af, 802eb57061d9457e817c3e700330b143; no voters:
I20260808 18:25:56.458338 9142 raft_consensus.cc:2781] T e3174a6300974b36bc1811929401b8fa P 0df99bbbc2c14b959b7f0a90656988af [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:25:56.458564 9142 raft_consensus.cc:686] T e3174a6300974b36bc1811929401b8fa P 0df99bbbc2c14b959b7f0a90656988af [term 1 LEADER]: Becoming Leader. State: Replica: 0df99bbbc2c14b959b7f0a90656988af, State: Running, Role: LEADER
I20260808 18:25:56.458915 9142 consensus_queue.cc:237] T e3174a6300974b36bc1811929401b8fa P 0df99bbbc2c14b959b7f0a90656988af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df99bbbc2c14b959b7f0a90656988af" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 34733 } } peers { permanent_uuid: "802eb57061d9457e817c3e700330b143" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 34819 } } peers { permanent_uuid: "69302204002a49cea7f3730f16da5a25" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 36963 } }
I20260808 18:25:56.462320 8632 catalog_manager.cc:5495] T e3174a6300974b36bc1811929401b8fa P 0df99bbbc2c14b959b7f0a90656988af reported cstate change: term changed from 0 to 1, leader changed from <none> to 0df99bbbc2c14b959b7f0a90656988af (127.23.72.131). New cstate: current_term: 1 leader_uuid: "0df99bbbc2c14b959b7f0a90656988af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df99bbbc2c14b959b7f0a90656988af" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 34733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "802eb57061d9457e817c3e700330b143" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 34819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69302204002a49cea7f3730f16da5a25" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 36963 } health_report { overall_health: UNKNOWN } } }
I20260808 18:25:56.497771 9135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:56.494539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42208 (local address 127.23.72.131:34733)
0808 18:25:56.494675 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 18:25:56.494679 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:25:56.494692 (+ 13us) server_negotiation.cc:407] Connection header received
0808 18:25:56.494766 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:25:56.494768 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:25:56.494824 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:25:56.494914 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:25:56.495141 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:56.495665 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:56.496477 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:25:56.496690 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:25:56.496766 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:25:56.497034 (+ 268us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:25:56.497465 (+ 431us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:25:56.497556 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 18:25:56.497615 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 18:25:56.497662 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 18:25:56.503860 8936 raft_consensus.cc:1261] T e3174a6300974b36bc1811929401b8fa P 802eb57061d9457e817c3e700330b143 [term 1 FOLLOWER]: Refusing update from remote peer 0df99bbbc2c14b959b7f0a90656988af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:25:56.503857 8783 raft_consensus.cc:1261] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25 [term 1 FOLLOWER]: Refusing update from remote peer 0df99bbbc2c14b959b7f0a90656988af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:25:56.504635 9142 consensus_queue.cc:1035] T e3174a6300974b36bc1811929401b8fa P 0df99bbbc2c14b959b7f0a90656988af [LEADER]: Connected to new peer: Peer: permanent_uuid: "69302204002a49cea7f3730f16da5a25" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 36963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:25:56.504868 9148 consensus_queue.cc:1035] T e3174a6300974b36bc1811929401b8fa P 0df99bbbc2c14b959b7f0a90656988af [LEADER]: Connected to new peer: Peer: permanent_uuid: "802eb57061d9457e817c3e700330b143" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 34819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:25:56.511063 9153 mvcc.cc:205] Tried to move back new op lower bound from 7316330727431966720 to 7316330727259123712. Current Snapshot: MvccSnapshot[applied={T|T < 7316330727431966720}]
I20260808 18:25:56.512223 9154 mvcc.cc:205] Tried to move back new op lower bound from 7316330727431966720 to 7316330727259123712. Current Snapshot: MvccSnapshot[applied={T|T < 7316330727431966720}]
I20260808 18:25:56.512826 9155 mvcc.cc:205] Tried to move back new op lower bound from 7316330727431966720 to 7316330727259123712. Current Snapshot: MvccSnapshot[applied={T|T < 7316330727431966720}]
I20260808 18:25:56.524029 8632 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40012:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:25:56.524230 8632 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40012:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:25:56.526785 8632 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d: Sending DeleteTablet for 3 replicas of tablet e3174a6300974b36bc1811929401b8fa
I20260808 18:25:56.527511 9056 tablet_service.cc:1514] Processing DeleteTablet for tablet e3174a6300974b36bc1811929401b8fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:25:56 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:42194
I20260808 18:25:56.527619 8763 tablet_service.cc:1514] Processing DeleteTablet for tablet e3174a6300974b36bc1811929401b8fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:25:56 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:50514
I20260808 18:25:56.527678 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 8674
I20260808 18:25:56.530148 8914 tablet_service.cc:1514] Processing DeleteTablet for tablet e3174a6300974b36bc1811929401b8fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:25:56 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:53788
I20260808 18:25:56.530261 9163 tablet_replica.cc:330] T e3174a6300974b36bc1811929401b8fa P 0df99bbbc2c14b959b7f0a90656988af: stopping tablet replica
I20260808 18:25:56.530674 9164 tablet_replica.cc:330] T e3174a6300974b36bc1811929401b8fa P 802eb57061d9457e817c3e700330b143: stopping tablet replica
I20260808 18:25:56.530782 9163 raft_consensus.cc:2229] T e3174a6300974b36bc1811929401b8fa P 0df99bbbc2c14b959b7f0a90656988af [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:25:56.531038 9164 raft_consensus.cc:2229] T e3174a6300974b36bc1811929401b8fa P 802eb57061d9457e817c3e700330b143 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:25:56.531279 9163 raft_consensus.cc:2258] T e3174a6300974b36bc1811929401b8fa P 0df99bbbc2c14b959b7f0a90656988af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:25:56.531368 9164 raft_consensus.cc:2258] T e3174a6300974b36bc1811929401b8fa P 802eb57061d9457e817c3e700330b143 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:25:56.532910 9163 ts_tablet_manager.cc:1905] T e3174a6300974b36bc1811929401b8fa P 0df99bbbc2c14b959b7f0a90656988af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:25:56.532910 9164 ts_tablet_manager.cc:1905] T e3174a6300974b36bc1811929401b8fa P 802eb57061d9457e817c3e700330b143: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:25:56.535830 9163 ts_tablet_manager.cc:1918] T e3174a6300974b36bc1811929401b8fa P 0df99bbbc2c14b959b7f0a90656988af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:25:56.535861 9164 ts_tablet_manager.cc:1918] T e3174a6300974b36bc1811929401b8fa P 802eb57061d9457e817c3e700330b143: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:25:56.535976 9163 log.cc:1196] T e3174a6300974b36bc1811929401b8fa P 0df99bbbc2c14b959b7f0a90656988af: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-2/wal/wals/e3174a6300974b36bc1811929401b8fa
I20260808 18:25:56.536002 9164 log.cc:1196] T e3174a6300974b36bc1811929401b8fa P 802eb57061d9457e817c3e700330b143: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-1/wal/wals/e3174a6300974b36bc1811929401b8fa
I20260808 18:25:56.536417 9163 ts_tablet_manager.cc:1939] T e3174a6300974b36bc1811929401b8fa P 0df99bbbc2c14b959b7f0a90656988af: Deleting consensus metadata
I20260808 18:25:56.536417 9164 ts_tablet_manager.cc:1939] T e3174a6300974b36bc1811929401b8fa P 802eb57061d9457e817c3e700330b143: Deleting consensus metadata
I20260808 18:25:56.537669 8619 catalog_manager.cc:4841] TS 0df99bbbc2c14b959b7f0a90656988af (127.23.72.131:34733): tablet e3174a6300974b36bc1811929401b8fa (table test-table [id=1fd1c4033e304ec19f59b09a478426a1]) successfully deleted
I20260808 18:25:56.537899 8619 catalog_manager.cc:4841] TS 802eb57061d9457e817c3e700330b143 (127.23.72.130:34819): tablet e3174a6300974b36bc1811929401b8fa (table test-table [id=1fd1c4033e304ec19f59b09a478426a1]) successfully deleted
W20260808 18:25:56.539383 8619 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.72.129:36963 (error 108)
I20260808 18:25:56.539582 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 8839
I20260808 18:25:56.541586 9134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:25:56.541296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:36963 (local address 127.0.0.1:50528)
0808 18:25:56.541432 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 18:25:56.541514 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.129:36963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 18:25:56.541782 8619 catalog_manager.cc:4584] TS 69302204002a49cea7f3730f16da5a25 (127.23.72.129:36963): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e3174a6300974b36bc1811929401b8fa: Network error: Client connection negotiation failed: client connection to 127.23.72.129:36963: connect: Connection refused (error 111)
I20260808 18:25:56.552225 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 8986
I20260808 18:25:56.559736 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 8599
I20260808 18:26:16.567515 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:46191
--webserver_interface=127.23.72.190
--webserver_port=42271
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:45461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:46191
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:26:16.682312 9166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:26:16.682576 9166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:26:16.682654 9166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:26:16.686718 9166 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 18:26:16.686775 9166 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 18:26:16.686791 9166 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 18:26:16.686805 9166 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:26:16.686825 9166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:26:16.686877 9166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:26:16.686909 9166 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:26:16.686930 9166 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:26:16.692917 9166 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:45461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:46191
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:46191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.72.190
--webserver_port=42271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:26:16.694043 9166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:26:16.695147 9166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:26:16.703944 9172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:16.703900 9166 server_base.cc:1034] running on GCE node
W20260808 18:26:16.703917 9173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:16.703984 9175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:16.704706 9166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:26:16.705854 9166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:26:16.707051 9166 hybrid_clock.cc:648] HybridClock initialized: now 1786213576707038 us; error 27 us; skew 500 ppm
Aug 08 18:26:16 dist-test-slave-5k54 krb5kdc[8019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213576, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:26:16.709964 9166 init.cc:377] Logged in from keytab as kudu/127.23.72.190@KRBTEST.COM (short username kudu)
I20260808 18:26:16.710796 9166 webserver.cc:460] Webserver started at http://127.23.72.190:42271/ using document root <none> and password file <none>
I20260808 18:26:16.711304 9166 fs_manager.cc:359] Metadata directory not provided
I20260808 18:26:16.711362 9166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:26:16.715225 9166 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:26:16.717314 9183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:16.718529 9166 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 18:26:16.718628 9166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "42c149b9c53a4e80a854d8db25ac438d"
format_stamp: "Formatted at 2026-08-08 18:25:34 on dist-test-slave-5k54"
server_key: "ffc482c5a6c05b21ee579622e101f4e0"
server_key_iv: "89e73e4faaa2d7598ff1a669c1d02846"
server_key_version: "encryptionkey@0"
I20260808 18:26:16.719141 9166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:26:16.752130 9166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:26:16.755586 9166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:26:16.755746 9166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:26:16.762898 9166 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.190:46191
I20260808 18:26:16.763214 9239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.190:46191 every 8 connection(s)
I20260808 18:26:16.764011 9166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:26:16.765455 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 9166
I20260808 18:26:16.765911 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.129:36963
--local_ip_for_outbound_sockets=127.23.72.129
--tserver_master_addrs=127.23.72.190:46191
--webserver_port=45303
--webserver_interface=127.23.72.129
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab
--principal=kudu/127.23.72.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:45461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:26:16.768721 9240 sys_catalog.cc:263] Verifying existing consensus state
I20260808 18:26:16.770481 9240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d: Bootstrap starting.
I20260808 18:26:16.788319 9240 log.cc:824] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d: Log is configured to *not* fsync() on all Append() calls
I20260808 18:26:16.799551 9240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 18:26:16.799944 9240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d: Bootstrap complete.
I20260808 18:26:16.804102 9240 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c149b9c53a4e80a854d8db25ac438d" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 46191 } }
I20260808 18:26:16.804490 9240 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42c149b9c53a4e80a854d8db25ac438d, State: Initialized, Role: FOLLOWER
I20260808 18:26:16.805023 9240 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c149b9c53a4e80a854d8db25ac438d" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 46191 } }
I20260808 18:26:16.805171 9240 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:26:16.805259 9240 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:26:16.805373 9240 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [term 2 FOLLOWER]: Advancing to term 3
I20260808 18:26:16.807204 9240 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c149b9c53a4e80a854d8db25ac438d" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 46191 } }
I20260808 18:26:16.807586 9240 leader_election.cc:302] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42c149b9c53a4e80a854d8db25ac438d; no voters:
I20260808 18:26:16.807896 9240 leader_election.cc:288] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 18:26:16.808058 9244 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [term 3 FOLLOWER]: Leader election won for term 3
I20260808 18:26:16.808312 9244 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [term 3 LEADER]: Becoming Leader. State: Replica: 42c149b9c53a4e80a854d8db25ac438d, State: Running, Role: LEADER
I20260808 18:26:16.808691 9244 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c149b9c53a4e80a854d8db25ac438d" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 46191 } }
I20260808 18:26:16.809098 9240 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:26:16.810213 9245 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "42c149b9c53a4e80a854d8db25ac438d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c149b9c53a4e80a854d8db25ac438d" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 46191 } } }
I20260808 18:26:16.810334 9245 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [sys.catalog]: This master's current role is: LEADER
I20260808 18:26:16.810711 9246 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42c149b9c53a4e80a854d8db25ac438d. Latest consensus state: current_term: 3 leader_uuid: "42c149b9c53a4e80a854d8db25ac438d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c149b9c53a4e80a854d8db25ac438d" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 46191 } } }
I20260808 18:26:16.810886 9246 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d [sys.catalog]: This master's current role is: LEADER
I20260808 18:26:16.811196 9253 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:26:16.815651 9253 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:26:16.816172 9253 catalog_manager.cc:1259] Loaded cluster ID: cfbae75465cc4c53a9de1e6a0f1a7efa
I20260808 18:26:16.816248 9253 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:26:16.818037 9253 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:26:16.818630 9253 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d: Loaded TSK: 0
I20260808 18:26:16.818717 9253 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 42c149b9c53a4e80a854d8db25ac438d: Loaded TSK: 1
F20260808 18:26:16.819311 9253 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7ff8796bd9ad google::LogMessage::Fail() at ??:0
@ 0x7ff8796c088a google::LogMessage::SendToLog() at ??:0
@ 0x7ff8796bd4de google::LogMessage::Flush() at ??:0
@ 0x7ff8796bea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7ff87c38118d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7ff87c379f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7ff8840227bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7ff883ffc99c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7ff883ffd178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7ff88403edf8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7ff88405d82b std::function<>::operator()() at ??:0
@ 0x7ff883ffcb4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7ff883ffdf3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7ff883ffa579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7ff88403e640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7ff87b90f3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7ff87b90fc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7ff87b9113f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7ff87b900512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7ff87cf446db start_thread at ??:0
@ 0x7ff8781bb61f clone at ??:0
W20260808 18:26:16.913151 9242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:26:16.913491 9242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:26:16.913564 9242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:26:16.920280 9242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:26:16.920368 9242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:26:16.920459 9242 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.129
I20260808 18:26:16.928637 9242 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:45461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab
--principal=kudu/127.23.72.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.129:36963
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.72.129
--webserver_port=45303
--tserver_master_addrs=127.23.72.190:46191
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9242
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.129
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:26:16.930202 9242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:26:16.931684 9242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:26:16.943102 9292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:16.943146 9291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:16.943125 9294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:16.943866 9242 server_base.cc:1034] running on GCE node
I20260808 18:26:16.944334 9242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:26:16.945123 9242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:26:16.946333 9242 hybrid_clock.cc:648] HybridClock initialized: now 1786213576946300 us; error 39 us; skew 500 ppm
Aug 08 18:26:16 dist-test-slave-5k54 krb5kdc[8019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213576, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:26:16.950003 9242 init.cc:377] Logged in from keytab as kudu/127.23.72.129@KRBTEST.COM (short username kudu)
I20260808 18:26:16.951092 9242 webserver.cc:460] Webserver started at http://127.23.72.129:45303/ using document root <none> and password file <none>
I20260808 18:26:16.951848 9242 fs_manager.cc:359] Metadata directory not provided
I20260808 18:26:16.951948 9242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:26:16.957212 9242 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260808 18:26:16.960001 9302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:16.961570 9242 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:26:16.961711 9242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-0/wal
uuid: "69302204002a49cea7f3730f16da5a25"
format_stamp: "Formatted at 2026-08-08 18:25:34 on dist-test-slave-5k54"
server_key: "756f5900f5382ef341510d7ca5b9494f"
server_key_iv: "7f3f90e976bcfcb36899215563a6cfe9"
server_key_version: "encryptionkey@0"
I20260808 18:26:16.962461 9242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:26:16.976632 9242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:26:16.980417 9242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:26:16.980643 9242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:26:16.981356 9242 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:26:16.982758 9309 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 18:26:16.986872 9242 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 18:26:16.986950 9242 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260808 18:26:16.987023 9242 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 18:26:16.989255 9242 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 18:26:16.989323 9242 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.001s
I20260808 18:26:16.989744 9309 tablet_bootstrap.cc:492] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25: Bootstrap starting.
I20260808 18:26:17.000240 9242 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.129:36963
I20260808 18:26:17.000571 9422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.129:36963 every 8 connection(s)
I20260808 18:26:17.001415 9242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
I20260808 18:26:17.005896 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 9242
I20260808 18:26:17.006428 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.130:34819
--local_ip_for_outbound_sockets=127.23.72.130
--tserver_master_addrs=127.23.72.190:46191
--webserver_port=38803
--webserver_interface=127.23.72.130
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab
--principal=kudu/127.23.72.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:45461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:26:17.010217 9309 log.cc:824] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25: Log is configured to *not* fsync() on all Append() calls
I20260808 18:26:17.020640 9309 tablet_bootstrap.cc:492] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 18:26:17.021103 9309 tablet_bootstrap.cc:492] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25: Bootstrap complete.
I20260808 18:26:17.022612 9309 ts_tablet_manager.cc:1397] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25: Time spent bootstrapping tablet: real 0.033s user 0.017s sys 0.013s
I20260808 18:26:17.025877 9309 raft_consensus.cc:348] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df99bbbc2c14b959b7f0a90656988af" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 34733 } } peers { permanent_uuid: "802eb57061d9457e817c3e700330b143" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 34819 } } peers { permanent_uuid: "69302204002a49cea7f3730f16da5a25" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 36963 } }
I20260808 18:26:17.026427 9309 raft_consensus.cc:729] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69302204002a49cea7f3730f16da5a25, State: Initialized, Role: FOLLOWER
I20260808 18:26:17.026978 9309 consensus_queue.cc:260] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df99bbbc2c14b959b7f0a90656988af" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 34733 } } peers { permanent_uuid: "802eb57061d9457e817c3e700330b143" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 34819 } } peers { permanent_uuid: "69302204002a49cea7f3730f16da5a25" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 36963 } }
I20260808 18:26:17.028105 9309 ts_tablet_manager.cc:1428] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25: Time spent starting tablet: real 0.005s user 0.008s sys 0.000s
W20260808 18:26:17.073015 9425 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:26:17.003835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.129:56027)
0808 18:26:17.004475 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:17.004533 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:17.005739 (+ 1206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:17.072204 (+ 66465us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":192,"threads_started":1}
W20260808 18:26:17.073618 9423 heartbeater.cc:644] Failed to heartbeat to 127.23.72.190:46191 (0 consecutive failures): Network error: Failed to ping master at 127.23.72.190:46191: Client connection negotiation failed: client connection to 127.23.72.190:46191: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 18:26:17.074429 9425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:17.074163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.129:39097)
0808 18:26:17.074292 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:17.074370 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260808 18:26:17.075150 9425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:17.074909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.129:41321)
0808 18:26:17.075045 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:17.075095 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260808 18:26:17.075511 9423 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 18:26:17.128569 9426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:26:17.128829 9426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:26:17.128873 9426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:26:17.133034 9426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:26:17.133091 9426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:26:17.133149 9426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.130
I20260808 18:26:17.138345 9426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:45461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab
--principal=kudu/127.23.72.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.130:34819
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.72.130
--webserver_port=38803
--tserver_master_addrs=127.23.72.190:46191
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.130
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:26:17.139531 9426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:26:17.140524 9426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:26:17.150122 9433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:17.150132 9434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:17.150183 9426 server_base.cc:1034] running on GCE node
W20260808 18:26:17.150133 9436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:17.150691 9426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:26:17.151352 9426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:26:17.152535 9426 hybrid_clock.cc:648] HybridClock initialized: now 1786213577152503 us; error 45 us; skew 500 ppm
Aug 08 18:26:17 dist-test-slave-5k54 krb5kdc[8019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213577, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:26:17.155411 9426 init.cc:377] Logged in from keytab as kudu/127.23.72.130@KRBTEST.COM (short username kudu)
I20260808 18:26:17.156323 9426 webserver.cc:460] Webserver started at http://127.23.72.130:38803/ using document root <none> and password file <none>
I20260808 18:26:17.156876 9426 fs_manager.cc:359] Metadata directory not provided
I20260808 18:26:17.156937 9426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:26:17.160674 9426 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260808 18:26:17.162674 9443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:17.163973 9426 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 18:26:17.164103 9426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-1/wal
uuid: "802eb57061d9457e817c3e700330b143"
format_stamp: "Formatted at 2026-08-08 18:25:34 on dist-test-slave-5k54"
server_key: "27a71935d215f18387c4d896c480cbbc"
server_key_iv: "715815ef6fedf61ff01271c330ae925d"
server_key_version: "encryptionkey@0"
I20260808 18:26:17.164644 9426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:26:17.179028 9426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:26:17.182310 9426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:26:17.182518 9426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:26:17.183135 9426 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:26:17.183987 9426 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:26:17.184057 9426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:17.184127 9426 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:26:17.184175 9426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:17.193097 9426 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.130:34819
I20260808 18:26:17.193135 9562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.130:34819 every 8 connection(s)
I20260808 18:26:17.194067 9426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
I20260808 18:26:17.195406 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 9426
I20260808 18:26:17.195869 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.131:34733
--local_ip_for_outbound_sockets=127.23.72.131
--tserver_master_addrs=127.23.72.190:46191
--webserver_port=33835
--webserver_interface=127.23.72.131
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab
--principal=kudu/127.23.72.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:45461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:26:17.198060 9565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:17.196254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.130:50279)
0808 18:26:17.196725 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:17.196852 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":144,"threads_started":1}
W20260808 18:26:17.198621 9563 heartbeater.cc:644] Failed to heartbeat to 127.23.72.190:46191 (0 consecutive failures): Network error: Failed to ping master at 127.23.72.190:46191: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
I20260808 18:26:17.199245 9565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:17.199027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.130:51889)
0808 18:26:17.199142 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:17.199194 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 18:26:17.199980 9565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:17.199788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.130:50425)
0808 18:26:17.199889 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:17.199934 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 18:26:17.200201 9563 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 18:26:17.308406 9566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:26:17.308661 9566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:26:17.308737 9566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:26:17.312727 9566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:26:17.312776 9566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:26:17.312865 9566 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.131
I20260808 18:26:17.318003 9566 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:45461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab
--principal=kudu/127.23.72.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.131:34733
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.72.131
--webserver_port=33835
--tserver_master_addrs=127.23.72.190:46191
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.131
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:26:17.318969 9566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:26:17.319967 9566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:26:17.329358 9574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:17.329499 9566 server_base.cc:1034] running on GCE node
W20260808 18:26:17.329358 9573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:17.329401 9576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:17.330056 9566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:26:17.330677 9566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:26:17.331832 9566 hybrid_clock.cc:648] HybridClock initialized: now 1786213577331817 us; error 26 us; skew 500 ppm
Aug 08 18:26:17 dist-test-slave-5k54 krb5kdc[8019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213577, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:26:17.334527 9566 init.cc:377] Logged in from keytab as kudu/127.23.72.131@KRBTEST.COM (short username kudu)
I20260808 18:26:17.335460 9566 webserver.cc:460] Webserver started at http://127.23.72.131:33835/ using document root <none> and password file <none>
I20260808 18:26:17.336035 9566 fs_manager.cc:359] Metadata directory not provided
I20260808 18:26:17.336129 9566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:26:17.339877 9566 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 18:26:17.341889 9583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:17.343080 9566 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:26:17.343214 9566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-2/wal
uuid: "0df99bbbc2c14b959b7f0a90656988af"
format_stamp: "Formatted at 2026-08-08 18:25:35 on dist-test-slave-5k54"
server_key: "66d1bf5677d875d05f11dce131dc72bc"
server_key_iv: "8428f367a03d10657fdf1b3da9a8eb69"
server_key_version: "encryptionkey@0"
I20260808 18:26:17.343808 9566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:26:17.382668 9566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:26:17.386212 9566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:26:17.386431 9566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:26:17.387066 9566 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:26:17.387935 9566 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:26:17.388015 9566 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:17.388090 9566 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:26:17.388144 9566 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:17.397347 9566 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.131:34733
I20260808 18:26:17.397378 9701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.131:34733 every 8 connection(s)
I20260808 18:26:17.398324 9566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
I20260808 18:26:17.402103 9704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:17.400394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.131:40827)
0808 18:26:17.400851 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:17.400963 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":113,"threads_started":1}
W20260808 18:26:17.402603 9702 heartbeater.cc:644] Failed to heartbeat to 127.23.72.190:46191 (0 consecutive failures): Network error: Failed to ping master at 127.23.72.190:46191: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
I20260808 18:26:17.403156 9704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:17.402956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.131:46219)
0808 18:26:17.403058 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:17.403104 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 18:26:17.403858 9704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:17.403613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.131:53255)
0808 18:26:17.403733 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:17.403787 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 18:26:17.403980 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 9566
W20260808 18:26:17.404137 9702 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 18:26:18.076545 9706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:18.076100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.129:56559)
0808 18:26:18.076372 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:18.076430 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":75,"threads_started":1}
I20260808 18:26:18.201066 9707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:18.200689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.130:34679)
0808 18:26:18.200918 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:18.200968 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":69,"threads_started":1}
I20260808 18:26:18.405027 9708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:18.404643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.131:56677)
0808 18:26:18.404876 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:18.404930 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":68,"threads_started":1}
I20260808 18:26:18.807837 9709 raft_consensus.cc:482] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:26:18.808089 9709 raft_consensus.cc:504] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df99bbbc2c14b959b7f0a90656988af" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 34733 } } peers { permanent_uuid: "802eb57061d9457e817c3e700330b143" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 34819 } } peers { permanent_uuid: "69302204002a49cea7f3730f16da5a25" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 36963 } }
I20260808 18:26:18.809173 9709 leader_election.cc:288] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0df99bbbc2c14b959b7f0a90656988af (127.23.72.131:34733), 802eb57061d9457e817c3e700330b143 (127.23.72.130:34819)
Aug 08 18:26:18 dist-test-slave-5k54 krb5kdc[8019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213576, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for kudu/127.23.72.131@KRBTEST.COM
Aug 08 18:26:18 dist-test-slave-5k54 krb5kdc[8019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213576, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for kudu/127.23.72.130@KRBTEST.COM
I20260808 18:26:18.823241 9712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:18.809807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.130:34819 (local address 127.23.72.129:52771)
0808 18:26:18.810056 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:18.810072 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:18.810173 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:18.811587 (+ 1414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:18.811593 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:18.812139 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:26:18.812639 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:18.812655 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:18.814042 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:18.814046 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:18.815004 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:18.815014 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:18.815793 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:18.815796 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:18.816014 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:18.816749 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:26:18.816760 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:26:18.818727 (+ 1967us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:26:18.820806 (+ 2079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:26:18.820810 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:26:18.820812 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:26:18.821145 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:26:18.821493 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:26:18.821495 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:26:18.821497 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:26:18.821651 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:26:18.822286 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:26:18.822289 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:26:18.822523 (+ 234us) client_negotiation.cc:769] Sending connection context
0808 18:26:18.822708 (+ 185us) client_negotiation.cc:240] Negotiation successful
0808 18:26:18.823052 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":199,"spinlock_wait_cycles":44032,"thread_start_us":63,"threads_started":1}
I20260808 18:26:18.823290 9711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:18.809576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:41697 (local address 127.23.72.131:34733)
0808 18:26:18.810046 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:18.810075 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:18.810125 (+ 50us) server_negotiation.cc:407] Connection header received
0808 18:26:18.810837 (+ 712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:18.810842 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:18.811193 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:18.811490 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:18.812860 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:18.813879 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:18.815134 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:18.815664 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:18.815851 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:18.818612 (+ 2761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:18.818618 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:18.818629 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:26:18.818659 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:18.820633 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:18.821255 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:18.821267 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:18.821273 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:18.821387 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:18.821764 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:18.821768 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:18.821770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:18.822161 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:18.822303 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:18.822818 (+ 515us) server_negotiation.cc:299] Negotiation successful
0808 18:26:18.823104 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":109,"threads_started":1}
I20260808 18:26:18.824036 9710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:18.809422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.131:34733 (local address 127.23.72.129:41697)
0808 18:26:18.809661 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:18.809677 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:18.809781 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:18.811588 (+ 1807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:18.811594 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:18.812140 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:26:18.812636 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:18.812652 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:18.814050 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:18.814052 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:18.815008 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:18.815017 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:18.815789 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:18.815792 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:18.816045 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:18.816698 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:26:18.816714 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:26:18.818365 (+ 1651us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:26:18.820771 (+ 2406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:26:18.820777 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:26:18.820790 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:26:18.821134 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:26:18.821489 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:26:18.821494 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:26:18.821500 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:26:18.821663 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:26:18.822282 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:26:18.822288 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:26:18.822523 (+ 235us) client_negotiation.cc:769] Sending connection context
0808 18:26:18.822708 (+ 185us) client_negotiation.cc:240] Negotiation successful
0808 18:26:18.823139 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":338,"spinlock_wait_cycles":32000,"thread_start_us":86,"threads_started":1}
I20260808 18:26:18.824065 9713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:18.809999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:52771 (local address 127.23.72.130:34819)
0808 18:26:18.810311 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:18.810320 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:18.810351 (+ 31us) server_negotiation.cc:407] Connection header received
0808 18:26:18.810981 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:18.810987 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:18.811267 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:18.811588 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:18.812832 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:18.813886 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:18.815138 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:18.815664 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:18.815840 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:18.818910 (+ 3070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:18.818914 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:18.818931 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:26:18.818958 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:18.820686 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:18.821258 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:18.821264 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:18.821276 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:18.821386 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:18.821760 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:18.821763 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:18.821765 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:18.822164 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:18.822300 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:18.822817 (+ 517us) server_negotiation.cc:299] Negotiation successful
0808 18:26:18.823935 (+ 1118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":120,"threads_started":1}
W20260808 18:26:18.826268 9305 leader_election.cc:341] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 802eb57061d9457e817c3e700330b143 (127.23.72.130:34819): Not found: Tablet not found: e3174a6300974b36bc1811929401b8fa
W20260808 18:26:18.826489 9305 leader_election.cc:341] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0df99bbbc2c14b959b7f0a90656988af (127.23.72.131:34733): Not found: Tablet not found: e3174a6300974b36bc1811929401b8fa
I20260808 18:26:18.826555 9305 leader_election.cc:302] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 69302204002a49cea7f3730f16da5a25; no voters: 0df99bbbc2c14b959b7f0a90656988af, 802eb57061d9457e817c3e700330b143
I20260808 18:26:18.826866 9709 raft_consensus.cc:2726] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 18:26:19.077613 9710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:19.077306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.129:52729)
0808 18:26:19.077465 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:19.077523 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 18:26:19.202219 9714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:19.201804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.130:55881)
0808 18:26:19.202050 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:19.202108 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":74,"threads_started":1}
I20260808 18:26:19.406101 9715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:19.405720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.131:54871)
0808 18:26:19.405956 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:19.406014 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":71,"threads_started":1}
I20260808 18:26:20.078881 9716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:20.078470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.129:39107)
0808 18:26:20.078726 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:20.078783 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":81,"threads_started":1}
I20260808 18:26:20.203366 9717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:20.202966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.130:47381)
0808 18:26:20.203219 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:20.203272 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":78,"threads_started":1}
I20260808 18:26:20.407298 9718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:20.406849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.131:43803)
0808 18:26:20.407096 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:20.407181 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":76,"threads_started":1}
I20260808 18:26:20.853446 9719 raft_consensus.cc:482] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:26:20.853616 9719 raft_consensus.cc:504] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df99bbbc2c14b959b7f0a90656988af" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 34733 } } peers { permanent_uuid: "802eb57061d9457e817c3e700330b143" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 34819 } } peers { permanent_uuid: "69302204002a49cea7f3730f16da5a25" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 36963 } }
I20260808 18:26:20.854063 9719 leader_election.cc:288] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0df99bbbc2c14b959b7f0a90656988af (127.23.72.131:34733), 802eb57061d9457e817c3e700330b143 (127.23.72.130:34819)
W20260808 18:26:20.854987 9305 leader_election.cc:341] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0df99bbbc2c14b959b7f0a90656988af (127.23.72.131:34733): Not found: Tablet not found: e3174a6300974b36bc1811929401b8fa
W20260808 18:26:20.855118 9305 leader_election.cc:341] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 802eb57061d9457e817c3e700330b143 (127.23.72.130:34819): Not found: Tablet not found: e3174a6300974b36bc1811929401b8fa
I20260808 18:26:20.855192 9305 leader_election.cc:302] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 69302204002a49cea7f3730f16da5a25; no voters: 0df99bbbc2c14b959b7f0a90656988af, 802eb57061d9457e817c3e700330b143
I20260808 18:26:20.855466 9719 raft_consensus.cc:2726] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 18:26:21.080099 9721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:21.079669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.129:53799)
0808 18:26:21.079945 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:21.080002 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":95,"threads_started":1}
I20260808 18:26:21.204636 9722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:21.204178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.130:44919)
0808 18:26:21.204476 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:21.204540 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":94,"threads_started":1}
I20260808 18:26:21.408524 9723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:21.408114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.131:51133)
0808 18:26:21.408367 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:21.408428 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":73,"threads_started":1}
I20260808 18:26:22.081406 9724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:22.080959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.129:45771)
0808 18:26:22.081259 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:22.081310 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":95,"threads_started":1}
I20260808 18:26:22.205823 9725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:22.205402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.130:49431)
0808 18:26:22.205679 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:22.205735 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":118,"threads_started":1}
I20260808 18:26:22.409659 9726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:22.409255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.131:38655)
0808 18:26:22.409514 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:22.409568 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":92,"threads_started":1}
I20260808 18:26:23.082751 9727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:23.082263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.129:33453)
0808 18:26:23.082525 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:23.082591 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":97,"threads_started":1}
I20260808 18:26:23.206997 9728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:23.206556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.130:53277)
0808 18:26:23.206840 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:23.206899 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":97,"threads_started":1}
I20260808 18:26:23.410759 9729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:23.410349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.131:48417)
0808 18:26:23.410611 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:23.410667 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":74,"threads_started":1}
I20260808 18:26:24.064703 9730 raft_consensus.cc:482] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:26:24.064852 9730 raft_consensus.cc:504] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df99bbbc2c14b959b7f0a90656988af" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 34733 } } peers { permanent_uuid: "802eb57061d9457e817c3e700330b143" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 34819 } } peers { permanent_uuid: "69302204002a49cea7f3730f16da5a25" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 36963 } }
I20260808 18:26:24.065297 9730 leader_election.cc:288] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0df99bbbc2c14b959b7f0a90656988af (127.23.72.131:34733), 802eb57061d9457e817c3e700330b143 (127.23.72.130:34819)
W20260808 18:26:24.066123 9305 leader_election.cc:341] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0df99bbbc2c14b959b7f0a90656988af (127.23.72.131:34733): Not found: Tablet not found: e3174a6300974b36bc1811929401b8fa
W20260808 18:26:24.066237 9305 leader_election.cc:341] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 802eb57061d9457e817c3e700330b143 (127.23.72.130:34819): Not found: Tablet not found: e3174a6300974b36bc1811929401b8fa
I20260808 18:26:24.066274 9305 leader_election.cc:302] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 69302204002a49cea7f3730f16da5a25; no voters: 0df99bbbc2c14b959b7f0a90656988af, 802eb57061d9457e817c3e700330b143
I20260808 18:26:24.066411 9730 raft_consensus.cc:2726] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 18:26:24.084089 9731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:24.083702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.129:50597)
0808 18:26:24.083932 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:24.083997 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":71,"threads_started":1}
I20260808 18:26:24.208263 9732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:24.207765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.130:33917)
0808 18:26:24.208077 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:24.208134 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":93,"threads_started":1}
I20260808 18:26:24.412123 9735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:24.411621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.131:55565)
0808 18:26:24.411961 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:24.412022 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":94,"threads_started":1}
I20260808 18:26:25.085234 9736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:25.084829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.129:57201)
0808 18:26:25.085081 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:25.085135 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":76,"threads_started":1}
I20260808 18:26:25.209434 9737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:25.209025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.130:48185)
0808 18:26:25.209270 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:25.209327 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":74,"threads_started":1}
I20260808 18:26:25.413244 9738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:25.412836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.131:58407)
0808 18:26:25.413097 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:25.413151 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":92,"threads_started":1}
I20260808 18:26:26.086607 9739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:26.086085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.129:56295)
0808 18:26:26.086432 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:26.086505 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":169,"threads_started":1}
I20260808 18:26:26.210532 9740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:26.210135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.130:55107)
0808 18:26:26.210385 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:26.210440 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":80,"threads_started":1}
I20260808 18:26:26.295708 9741 raft_consensus.cc:482] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:26:26.295885 9741 raft_consensus.cc:504] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df99bbbc2c14b959b7f0a90656988af" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 34733 } } peers { permanent_uuid: "802eb57061d9457e817c3e700330b143" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 34819 } } peers { permanent_uuid: "69302204002a49cea7f3730f16da5a25" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 36963 } }
I20260808 18:26:26.296315 9741 leader_election.cc:288] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0df99bbbc2c14b959b7f0a90656988af (127.23.72.131:34733), 802eb57061d9457e817c3e700330b143 (127.23.72.130:34819)
W20260808 18:26:26.297314 9305 leader_election.cc:341] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0df99bbbc2c14b959b7f0a90656988af (127.23.72.131:34733): Not found: Tablet not found: e3174a6300974b36bc1811929401b8fa
W20260808 18:26:26.297468 9305 leader_election.cc:341] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 802eb57061d9457e817c3e700330b143 (127.23.72.130:34819): Not found: Tablet not found: e3174a6300974b36bc1811929401b8fa
I20260808 18:26:26.297526 9305 leader_election.cc:302] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 69302204002a49cea7f3730f16da5a25; no voters: 0df99bbbc2c14b959b7f0a90656988af, 802eb57061d9457e817c3e700330b143
I20260808 18:26:26.297714 9741 raft_consensus.cc:2726] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 18:26:26.414402 9742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:26.413965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.131:60749)
0808 18:26:26.414232 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:26.414307 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":79,"threads_started":1}
I20260808 18:26:27.087990 9743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:27.087572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.129:44177)
0808 18:26:27.087821 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:27.087876 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":73,"threads_started":1}
I20260808 18:26:27.211772 9744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:27.211294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.130:49015)
0808 18:26:27.211616 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:27.211678 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":160,"threads_started":1}
I20260808 18:26:27.415566 9745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:27.415095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.131:46195)
0808 18:26:27.415353 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:27.415406 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":90,"threads_started":1}
I20260808 18:26:28.089165 9746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:28.088746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.129:45461)
0808 18:26:28.089004 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:28.089064 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":79,"threads_started":1}
I20260808 18:26:28.212894 9747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:28.212481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.130:43481)
0808 18:26:28.212747 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:28.212801 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":96,"threads_started":1}
I20260808 18:26:28.416693 9748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:28.416250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.131:58513)
0808 18:26:28.416543 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:28.416600 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":98,"threads_started":1}
I20260808 18:26:29.090593 9749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:29.090055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.129:40915)
0808 18:26:29.090426 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:29.090492 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":114,"threads_started":1}
I20260808 18:26:29.214095 9750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:29.213656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.130:53537)
0808 18:26:29.213931 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:29.213997 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":96,"threads_started":1}
I20260808 18:26:29.417919 9751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:29.417489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.131:37421)
0808 18:26:29.417760 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:29.417825 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":94,"threads_started":1}
I20260808 18:26:30.091995 9752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:30.091567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.129:37651)
0808 18:26:30.091840 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:30.091894 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":97,"threads_started":1}
I20260808 18:26:30.215183 9753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:30.214787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.130:56223)
0808 18:26:30.215038 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:30.215090 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":88,"threads_started":1}
I20260808 18:26:30.398505 9754 raft_consensus.cc:482] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:26:30.398695 9754 raft_consensus.cc:504] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df99bbbc2c14b959b7f0a90656988af" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 34733 } } peers { permanent_uuid: "802eb57061d9457e817c3e700330b143" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 34819 } } peers { permanent_uuid: "69302204002a49cea7f3730f16da5a25" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 36963 } }
I20260808 18:26:30.399159 9754 leader_election.cc:288] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0df99bbbc2c14b959b7f0a90656988af (127.23.72.131:34733), 802eb57061d9457e817c3e700330b143 (127.23.72.130:34819)
W20260808 18:26:30.399945 9305 leader_election.cc:341] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 802eb57061d9457e817c3e700330b143 (127.23.72.130:34819): Not found: Tablet not found: e3174a6300974b36bc1811929401b8fa
W20260808 18:26:30.400105 9305 leader_election.cc:341] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0df99bbbc2c14b959b7f0a90656988af (127.23.72.131:34733): Not found: Tablet not found: e3174a6300974b36bc1811929401b8fa
I20260808 18:26:30.400166 9305 leader_election.cc:302] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 69302204002a49cea7f3730f16da5a25; no voters: 0df99bbbc2c14b959b7f0a90656988af, 802eb57061d9457e817c3e700330b143
I20260808 18:26:30.400328 9754 raft_consensus.cc:2726] T e3174a6300974b36bc1811929401b8fa P 69302204002a49cea7f3730f16da5a25 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 18:26:30.419027 9755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:30.418609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.131:38285)
0808 18:26:30.418886 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:30.418937 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":95,"threads_started":1}
I20260808 18:26:31.093179 9756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:31.092713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.129:36245)
0808 18:26:31.093023 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:31.093079 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":95,"threads_started":1}
I20260808 18:26:31.216329 9757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:31.215877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.130:59523)
0808 18:26:31.216171 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:31.216230 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":118,"threads_started":1}
I20260808 18:26:31.420172 9758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:31.419747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.131:48539)
0808 18:26:31.420017 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:31.420077 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":75,"threads_started":1}
I20260808 18:26:32.094538 9760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:32.094123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.129:53757)
0808 18:26:32.094387 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:32.094441 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":80,"threads_started":1}
I20260808 18:26:32.217643 9763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:32.217195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.130:35467)
0808 18:26:32.217487 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:32.217546 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":75,"threads_started":1}
I20260808 18:26:32.409127 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 9242
I20260808 18:26:32.417840 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 9426
I20260808 18:26:32.421629 9764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:32.421189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:46191 (local address 127.23.72.131:55681)
0808 18:26:32.421475 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:32.421531 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.190:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":87,"threads_started":1}
I20260808 18:26:32.425025 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 9566
2026-08-08T18:26:32Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59319 ms)
[----------] 30 tests from SecurityITest (246989 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:26:32 dist-test-slave-5k54 krb5kdc[9767](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:26:32 dist-test-slave-5k54 krb5kdc[9767](info): set up 2 sockets
Aug 08 18:26:32 dist-test-slave-5k54 krb5kdc[9767](info): commencing operation
krb5kdc: starting...
W20260808 18:26:33.530952 23842 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Aug 08 18:26:33 dist-test-slave-5k54 krb5kdc[9767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213593, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:26:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:26:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.72.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
I20260808 18:26:33.664456 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:41593
--webserver_interface=127.23.72.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:44783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:41593
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:26:33.775856 9780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:26:33.776117 9780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:26:33.776157 9780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:26:33.780131 9780 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:26:33.780182 9780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:26:33.780202 9780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:26:33.780218 9780 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:26:33.780279 9780 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:26:33.785482 9780 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:44783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:41593
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:41593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.72.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9780
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:26:33.786515 9780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:26:33.787555 9780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:26:33.796712 9786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:33.796748 9790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:33.796770 9780 server_base.cc:1034] running on GCE node
W20260808 18:26:33.796696 9788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:33.797598 9780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:26:33.798686 9780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:26:33.799906 9780 hybrid_clock.cc:648] HybridClock initialized: now 1786213593799881 us; error 41 us; skew 500 ppm
Aug 08 18:26:33 dist-test-slave-5k54 krb5kdc[9767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213593, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:26:33.802646 9780 init.cc:377] Logged in from keytab as kudu/127.23.72.190@KRBTEST.COM (short username kudu)
I20260808 18:26:33.803570 9780 webserver.cc:460] Webserver started at http://127.23.72.190:35465/ using document root <none> and password file <none>
I20260808 18:26:33.804087 9780 fs_manager.cc:359] Metadata directory not provided
I20260808 18:26:33.804143 9780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:26:33.804303 9780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:26:33.806264 9780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "1f3ab6be0e4c4d6294fe32fa8e3f1ada"
format_stamp: "Formatted at 2026-08-08 18:26:33 on dist-test-slave-5k54"
server_key: "1ebd9c3bc794b0717213a72660548396"
server_key_iv: "32a3a9425b95c10bc528eb308e5e6355"
server_key_version: "encryptionkey@0"
I20260808 18:26:33.806767 9780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "1f3ab6be0e4c4d6294fe32fa8e3f1ada"
format_stamp: "Formatted at 2026-08-08 18:26:33 on dist-test-slave-5k54"
server_key: "1ebd9c3bc794b0717213a72660548396"
server_key_iv: "32a3a9425b95c10bc528eb308e5e6355"
server_key_version: "encryptionkey@0"
I20260808 18:26:33.810456 9780 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260808 18:26:33.813098 9797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:33.814366 9780 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 18:26:33.814518 9780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "1f3ab6be0e4c4d6294fe32fa8e3f1ada"
format_stamp: "Formatted at 2026-08-08 18:26:33 on dist-test-slave-5k54"
server_key: "1ebd9c3bc794b0717213a72660548396"
server_key_iv: "32a3a9425b95c10bc528eb308e5e6355"
server_key_version: "encryptionkey@0"
I20260808 18:26:33.814636 9780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:26:33.826676 9780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:26:33.829591 9780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:26:33.829804 9780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:26:33.837031 9780 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.190:41593
I20260808 18:26:33.837060 9852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.190:41593 every 8 connection(s)
I20260808 18:26:33.838006 9780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:26:33.840742 9853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:26:33.840783 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 9780
I20260808 18:26:33.840947 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:26:33.841286 23842 external_mini_cluster.cc:1428] Setting key 3497b611edbe9a5b58398d0c4a7ea9bc
I20260808 18:26:33.847146 9853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f3ab6be0e4c4d6294fe32fa8e3f1ada: Bootstrap starting.
I20260808 18:26:33.849623 9853 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f3ab6be0e4c4d6294fe32fa8e3f1ada: Neither blocks nor log segments found. Creating new log.
Aug 08 18:26:33 dist-test-slave-5k54 krb5kdc[9767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213593, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:26:33.850447 9853 log.cc:824] T 00000000000000000000000000000000 P 1f3ab6be0e4c4d6294fe32fa8e3f1ada: Log is configured to *not* fsync() on all Append() calls
I20260808 18:26:33.852607 9853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f3ab6be0e4c4d6294fe32fa8e3f1ada: No bootstrap required, opened a new log
I20260808 18:26:33.855894 9857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:33.842740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33296 (local address 127.23.72.190:41593)
0808 18:26:33.843193 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:33.843208 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:33.843242 (+ 34us) server_negotiation.cc:407] Connection header received
0808 18:26:33.844026 (+ 784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:33.844046 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:33.844380 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:33.844721 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:33.845593 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:33.846586 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:33.847561 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:33.848033 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:33.848207 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:33.850770 (+ 2563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:33.850775 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:33.850814 (+ 39us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:26:33.850853 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:33.852740 (+ 1887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:33.853269 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:33.853274 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:33.853278 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:33.853417 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:33.853749 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:33.853752 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:33.853753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:33.854179 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:33.854333 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:33.854701 (+ 368us) server_negotiation.cc:299] Negotiation successful
0808 18:26:33.855051 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":141,"threads_started":1}
I20260808 18:26:33.856750 9853 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f3ab6be0e4c4d6294fe32fa8e3f1ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f3ab6be0e4c4d6294fe32fa8e3f1ada" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 41593 } }
I20260808 18:26:33.857048 9853 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f3ab6be0e4c4d6294fe32fa8e3f1ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:26:33.857102 9853 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f3ab6be0e4c4d6294fe32fa8e3f1ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f3ab6be0e4c4d6294fe32fa8e3f1ada, State: Initialized, Role: FOLLOWER
I20260808 18:26:33.857558 9853 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f3ab6be0e4c4d6294fe32fa8e3f1ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f3ab6be0e4c4d6294fe32fa8e3f1ada" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 41593 } }
I20260808 18:26:33.857717 9853 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f3ab6be0e4c4d6294fe32fa8e3f1ada [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:26:33.857781 9853 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f3ab6be0e4c4d6294fe32fa8e3f1ada [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:26:33.857877 9853 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f3ab6be0e4c4d6294fe32fa8e3f1ada [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:26:33.858776 9853 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f3ab6be0e4c4d6294fe32fa8e3f1ada [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f3ab6be0e4c4d6294fe32fa8e3f1ada" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 41593 } }
I20260808 18:26:33.859117 9853 leader_election.cc:302] T 00000000000000000000000000000000 P 1f3ab6be0e4c4d6294fe32fa8e3f1ada [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f3ab6be0e4c4d6294fe32fa8e3f1ada; no voters:
I20260808 18:26:33.859414 9853 leader_election.cc:288] T 00000000000000000000000000000000 P 1f3ab6be0e4c4d6294fe32fa8e3f1ada [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:26:33.859596 9859 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f3ab6be0e4c4d6294fe32fa8e3f1ada [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:26:33.859856 9859 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f3ab6be0e4c4d6294fe32fa8e3f1ada [term 1 LEADER]: Becoming Leader. State: Replica: 1f3ab6be0e4c4d6294fe32fa8e3f1ada, State: Running, Role: LEADER
I20260808 18:26:33.860314 9859 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f3ab6be0e4c4d6294fe32fa8e3f1ada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f3ab6be0e4c4d6294fe32fa8e3f1ada" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 41593 } }
I20260808 18:26:33.860699 9853 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f3ab6be0e4c4d6294fe32fa8e3f1ada [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:26:33.862041 9861 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f3ab6be0e4c4d6294fe32fa8e3f1ada [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f3ab6be0e4c4d6294fe32fa8e3f1ada. Latest consensus state: current_term: 1 leader_uuid: "1f3ab6be0e4c4d6294fe32fa8e3f1ada" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f3ab6be0e4c4d6294fe32fa8e3f1ada" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 41593 } } }
I20260808 18:26:33.862960 9860 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f3ab6be0e4c4d6294fe32fa8e3f1ada [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f3ab6be0e4c4d6294fe32fa8e3f1ada" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f3ab6be0e4c4d6294fe32fa8e3f1ada" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 41593 } } }
I20260808 18:26:33.862274 9861 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f3ab6be0e4c4d6294fe32fa8e3f1ada [sys.catalog]: This master's current role is: LEADER
I20260808 18:26:33.863852 9860 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f3ab6be0e4c4d6294fe32fa8e3f1ada [sys.catalog]: This master's current role is: LEADER
W20260808 18:26:33.865458 9873 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1f3ab6be0e4c4d6294fe32fa8e3f1ada: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:26:33.865525 9873 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:26:33.865609 9875 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:26:33.866297 9875 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:26:33.871634 9875 catalog_manager.cc:1347] Generated new cluster ID: bc8b6a88387747a2b54588902103d92c
I20260808 18:26:33.871706 9875 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:26:33.913787 9875 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:26:33.914750 9875 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:26:33.922724 9875 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f3ab6be0e4c4d6294fe32fa8e3f1ada: Generated new TSK 0
I20260808 18:26:33.923408 9875 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.72.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab.
Entry for principal kudu/127.23.72.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab.
I20260808 18:26:33.966956 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.129:0
--local_ip_for_outbound_sockets=127.23.72.129
--webserver_interface=127.23.72.129
--webserver_port=0
--tserver_master_addrs=127.23.72.190:41593
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab
--principal=kudu/127.23.72.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:44783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:26:34.082520 9880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:26:34.082803 9880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:26:34.082845 9880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:26:34.086776 9880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:26:34.086841 9880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:26:34.086898 9880 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.129
I20260808 18:26:34.092175 9880 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:44783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab
--principal=kudu/127.23.72.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.72.129
--webserver_port=0
--tserver_master_addrs=127.23.72.190:41593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9880
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.129
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:26:34.093159 9880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:26:34.094151 9880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:26:34.103713 9888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:34.103708 9885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:34.103839 9886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:34.104010 9880 server_base.cc:1034] running on GCE node
I20260808 18:26:34.104512 9880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:26:34.105189 9880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:26:34.106398 9880 hybrid_clock.cc:648] HybridClock initialized: now 1786213594106380 us; error 28 us; skew 500 ppm
Aug 08 18:26:34 dist-test-slave-5k54 krb5kdc[9767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213594, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:26:34.109326 9880 init.cc:377] Logged in from keytab as kudu/127.23.72.129@KRBTEST.COM (short username kudu)
I20260808 18:26:34.110183 9880 webserver.cc:460] Webserver started at http://127.23.72.129:46783/ using document root <none> and password file <none>
I20260808 18:26:34.110723 9880 fs_manager.cc:359] Metadata directory not provided
I20260808 18:26:34.110790 9880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:26:34.110956 9880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:26:34.113145 9880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-0/data/instance:
uuid: "eb12b77914b0429f827ef7bfb0a0b010"
format_stamp: "Formatted at 2026-08-08 18:26:34 on dist-test-slave-5k54"
server_key: "1aafbf25629792839366d515ec4db5ef"
server_key_iv: "152d9e60613d348e7a3d591799f83bd0"
server_key_version: "encryptionkey@0"
I20260808 18:26:34.113678 9880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance:
uuid: "eb12b77914b0429f827ef7bfb0a0b010"
format_stamp: "Formatted at 2026-08-08 18:26:34 on dist-test-slave-5k54"
server_key: "1aafbf25629792839366d515ec4db5ef"
server_key_iv: "152d9e60613d348e7a3d591799f83bd0"
server_key_version: "encryptionkey@0"
I20260808 18:26:34.117435 9880 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260808 18:26:34.120026 9895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:34.121351 9880 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:26:34.121446 9880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-0/wal
uuid: "eb12b77914b0429f827ef7bfb0a0b010"
format_stamp: "Formatted at 2026-08-08 18:26:34 on dist-test-slave-5k54"
server_key: "1aafbf25629792839366d515ec4db5ef"
server_key_iv: "152d9e60613d348e7a3d591799f83bd0"
server_key_version: "encryptionkey@0"
I20260808 18:26:34.121531 9880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:26:34.134150 9880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:26:34.137425 9880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:26:34.137578 9880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:26:34.138106 9880 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:26:34.138942 9880 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:26:34.138988 9880 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:34.139027 9880 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:26:34.139042 9880 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:34.148247 9880 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.129:35811
I20260808 18:26:34.148372 10017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.129:35811 every 8 connection(s)
I20260808 18:26:34.149222 9880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
I20260808 18:26:34.155172 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 9880
I20260808 18:26:34.155339 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance
I20260808 18:26:34.155700 23842 external_mini_cluster.cc:1428] Setting key 3085950f48bdb8a9b94cff3fc6679fc5
Aug 08 18:26:34 dist-test-slave-5k54 krb5kdc[9767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213594, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:26:34.165058 9857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:34.151181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:60691 (local address 127.23.72.190:41593)
0808 18:26:34.151351 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:34.151355 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:34.152188 (+ 833us) server_negotiation.cc:407] Connection header received
0808 18:26:34.153616 (+ 1428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:34.153618 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:34.153665 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:34.153740 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:34.155113 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:34.155885 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:34.157432 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:34.157696 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:34.157846 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:34.160662 (+ 2816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:34.160666 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:34.160668 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:26:34.160697 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:34.162488 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:34.163133 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:34.163135 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:34.163137 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:34.163224 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:34.163670 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:34.163673 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:34.163675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:34.163926 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:34.164026 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:34.164809 (+ 783us) server_negotiation.cc:299] Negotiation successful
0808 18:26:34.164929 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 18:26:34.166288 10020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:34.151524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41593 (local address 127.23.72.129:60691)
0808 18:26:34.152010 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:34.152057 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:34.153406 (+ 1349us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:34.153918 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:34.153925 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:34.154370 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:26:34.154927 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:34.154943 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:34.156034 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:34.156041 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:34.157294 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:34.157305 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:34.157902 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:34.157907 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:34.158091 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:34.158767 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:26:34.158785 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:26:34.160490 (+ 1705us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:26:34.162605 (+ 2115us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:26:34.162615 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:26:34.162635 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:26:34.163025 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:26:34.163327 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:26:34.163330 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:26:34.163332 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:26:34.163554 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:26:34.164059 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:26:34.164070 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:26:34.164456 (+ 386us) client_negotiation.cc:769] Sending connection context
0808 18:26:34.164702 (+ 246us) client_negotiation.cc:240] Negotiation successful
0808 18:26:34.165131 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":117,"threads_started":1}
I20260808 18:26:34.167706 10018 heartbeater.cc:347] Connected to a master server at 127.23.72.190:41593
WARNING: no policy specified for kudu/127.23.72.130@KRBTEST.COM; defaulting to no policy
I20260808 18:26:34.167987 10018 heartbeater.cc:464] Registering TS with master...
I20260808 18:26:34.168524 10018 heartbeater.cc:511] Master 127.23.72.190:41593 requested a full tablet report, sending...
I20260808 18:26:34.170100 9816 ts_manager.cc:194] Registered new tserver with Master: eb12b77914b0429f827ef7bfb0a0b010 (127.23.72.129:35811)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.130@KRBTEST.COM" created.
I20260808 18:26:34.171315 9816 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.129@KRBTEST.COM'} at 127.23.72.129:60691
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab.
Entry for principal kudu/127.23.72.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab.
I20260808 18:26:34.186830 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.130:0
--local_ip_for_outbound_sockets=127.23.72.130
--webserver_interface=127.23.72.130
--webserver_port=0
--tserver_master_addrs=127.23.72.190:41593
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab
--principal=kudu/127.23.72.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:44783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:26:34.299335 10025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:26:34.299856 10025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:26:34.299953 10025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:26:34.304150 10025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:26:34.304205 10025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:26:34.304293 10025 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.130
I20260808 18:26:34.309580 10025 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:44783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab
--principal=kudu/127.23.72.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.72.130
--webserver_port=0
--tserver_master_addrs=127.23.72.190:41593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10025
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.130
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:26:34.310662 10025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:26:34.311761 10025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:26:34.321643 10034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:34.321645 10031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:34.321671 10032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:34.322091 10025 server_base.cc:1034] running on GCE node
I20260808 18:26:34.322466 10025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:26:34.323220 10025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:26:34.324378 10025 hybrid_clock.cc:648] HybridClock initialized: now 1786213594324365 us; error 70 us; skew 500 ppm
Aug 08 18:26:34 dist-test-slave-5k54 krb5kdc[9767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213594, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:26:34.327312 10025 init.cc:377] Logged in from keytab as kudu/127.23.72.130@KRBTEST.COM (short username kudu)
I20260808 18:26:34.328266 10025 webserver.cc:460] Webserver started at http://127.23.72.130:33313/ using document root <none> and password file <none>
I20260808 18:26:34.328787 10025 fs_manager.cc:359] Metadata directory not provided
I20260808 18:26:34.328848 10025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:26:34.329022 10025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:26:34.331008 10025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-1/data/instance:
uuid: "865620b3ad1c4cd8a9b7b9806f9e1ede"
format_stamp: "Formatted at 2026-08-08 18:26:34 on dist-test-slave-5k54"
server_key: "6c344a4c4e38346fe98cdc453c5c15e2"
server_key_iv: "8f193571714572a30968a9704148ba85"
server_key_version: "encryptionkey@0"
I20260808 18:26:34.331600 10025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance:
uuid: "865620b3ad1c4cd8a9b7b9806f9e1ede"
format_stamp: "Formatted at 2026-08-08 18:26:34 on dist-test-slave-5k54"
server_key: "6c344a4c4e38346fe98cdc453c5c15e2"
server_key_iv: "8f193571714572a30968a9704148ba85"
server_key_version: "encryptionkey@0"
I20260808 18:26:34.335332 10025 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:26:34.337797 10043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:34.339080 10025 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:26:34.339214 10025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-1/wal
uuid: "865620b3ad1c4cd8a9b7b9806f9e1ede"
format_stamp: "Formatted at 2026-08-08 18:26:34 on dist-test-slave-5k54"
server_key: "6c344a4c4e38346fe98cdc453c5c15e2"
server_key_iv: "8f193571714572a30968a9704148ba85"
server_key_version: "encryptionkey@0"
I20260808 18:26:34.339326 10025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:26:34.369046 10025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:26:34.372437 10025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:26:34.372648 10025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:26:34.373260 10025 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:26:34.374152 10025 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:26:34.374222 10025 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:34.374293 10025 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:26:34.374346 10025 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:34.383682 10025 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.130:37291
I20260808 18:26:34.383713 10163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.130:37291 every 8 connection(s)
I20260808 18:26:34.384701 10025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
I20260808 18:26:34.394359 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 10025
I20260808 18:26:34.394517 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance
Aug 08 18:26:34 dist-test-slave-5k54 krb5kdc[9767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213594, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.130@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:26:34.394928 23842 external_mini_cluster.cc:1428] Setting key 461e606664121e45c3a6f66f16763fc8
I20260808 18:26:34.399299 9857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:34.386713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:36759 (local address 127.23.72.190:41593)
0808 18:26:34.386914 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:34.386921 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:34.387840 (+ 919us) server_negotiation.cc:407] Connection header received
0808 18:26:34.389147 (+ 1307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:34.389150 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:34.389203 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:34.389288 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:34.390489 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:34.391138 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:34.392345 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:34.392590 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:34.392683 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:34.395327 (+ 2644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:34.395333 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:34.395340 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:26:34.395377 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:34.397085 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:34.397669 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:34.397674 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:34.397676 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:34.397771 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:34.398149 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:34.398154 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:34.398156 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:34.398357 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:34.398475 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:34.398984 (+ 509us) server_negotiation.cc:299] Negotiation successful
0808 18:26:34.399118 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 18:26:34.400240 10166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:34.387157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41593 (local address 127.23.72.130:36759)
0808 18:26:34.387673 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:34.387716 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:34.388935 (+ 1219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:34.389450 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:34.389457 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:34.389812 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:26:34.390310 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:34.390324 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:34.391258 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:34.391261 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:34.392216 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:34.392226 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:34.392728 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:34.392731 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:34.392867 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:34.393497 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:26:34.393513 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:26:34.395133 (+ 1620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:26:34.397247 (+ 2114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:26:34.397254 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:26:34.397270 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:26:34.397552 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:26:34.397889 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:26:34.397893 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:26:34.397895 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:26:34.398043 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:26:34.398473 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:26:34.398481 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:26:34.398732 (+ 251us) client_negotiation.cc:769] Sending connection context
0808 18:26:34.398940 (+ 208us) client_negotiation.cc:240] Negotiation successful
0808 18:26:34.399275 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":126,"threads_started":1}
I20260808 18:26:34.401508 10164 heartbeater.cc:347] Connected to a master server at 127.23.72.190:41593
I20260808 18:26:34.401787 10164 heartbeater.cc:464] Registering TS with master...
I20260808 18:26:34.402323 10164 heartbeater.cc:511] Master 127.23.72.190:41593 requested a full tablet report, sending...
I20260808 18:26:34.403450 9816 ts_manager.cc:194] Registered new tserver with Master: 865620b3ad1c4cd8a9b7b9806f9e1ede (127.23.72.130:37291)
I20260808 18:26:34.404268 9816 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.130@KRBTEST.COM'} at 127.23.72.130:36759
WARNING: no policy specified for kudu/127.23.72.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab.
Entry for principal kudu/127.23.72.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab.
I20260808 18:26:34.424589 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.131:0
--local_ip_for_outbound_sockets=127.23.72.131
--webserver_interface=127.23.72.131
--webserver_port=0
--tserver_master_addrs=127.23.72.190:41593
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab
--principal=kudu/127.23.72.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:44783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:26:34.537983 10169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:26:34.538228 10169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:26:34.538269 10169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:26:34.542351 10169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:26:34.542404 10169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:26:34.542469 10169 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.131
I20260808 18:26:34.547668 10169 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:44783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab
--principal=kudu/127.23.72.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.72.131
--webserver_port=0
--tserver_master_addrs=127.23.72.190:41593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.131
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:26:34.548617 10169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:26:34.549573 10169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:26:34.561648 10177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:34.561621 10175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:34.561789 10174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:34.561882 10169 server_base.cc:1034] running on GCE node
I20260808 18:26:34.562463 10169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:26:34.563352 10169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:26:34.564574 10169 hybrid_clock.cc:648] HybridClock initialized: now 1786213594564558 us; error 34 us; skew 500 ppm
Aug 08 18:26:34 dist-test-slave-5k54 krb5kdc[9767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213594, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:26:34.567509 10169 init.cc:377] Logged in from keytab as kudu/127.23.72.131@KRBTEST.COM (short username kudu)
I20260808 18:26:34.568351 10169 webserver.cc:460] Webserver started at http://127.23.72.131:43813/ using document root <none> and password file <none>
I20260808 18:26:34.568939 10169 fs_manager.cc:359] Metadata directory not provided
I20260808 18:26:34.569036 10169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:26:34.569265 10169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:26:34.571270 10169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-2/data/instance:
uuid: "92188143c6d24314806938fd8a4269ed"
format_stamp: "Formatted at 2026-08-08 18:26:34 on dist-test-slave-5k54"
server_key: "025d8e7668dd373da2c927479ab760ff"
server_key_iv: "fcefe772d0ba540ee4c3e60adf2706e4"
server_key_version: "encryptionkey@0"
I20260808 18:26:34.571889 10169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance:
uuid: "92188143c6d24314806938fd8a4269ed"
format_stamp: "Formatted at 2026-08-08 18:26:34 on dist-test-slave-5k54"
server_key: "025d8e7668dd373da2c927479ab760ff"
server_key_iv: "fcefe772d0ba540ee4c3e60adf2706e4"
server_key_version: "encryptionkey@0"
I20260808 18:26:34.575591 10169 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 18:26:34.577966 10186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:34.579242 10169 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:26:34.579388 10169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-2/wal
uuid: "92188143c6d24314806938fd8a4269ed"
format_stamp: "Formatted at 2026-08-08 18:26:34 on dist-test-slave-5k54"
server_key: "025d8e7668dd373da2c927479ab760ff"
server_key_iv: "fcefe772d0ba540ee4c3e60adf2706e4"
server_key_version: "encryptionkey@0"
I20260808 18:26:34.579581 10169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:26:34.602094 10169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:26:34.605193 10169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:26:34.605396 10169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:26:34.605983 10169 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:26:34.606786 10169 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:26:34.606863 10169 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:34.606953 10169 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:26:34.607005 10169 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:34.615856 10169 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.131:37519
I20260808 18:26:34.615906 10313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.131:37519 every 8 connection(s)
I20260808 18:26:34.616798 10169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
I20260808 18:26:34.621980 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 10169
I20260808 18:26:34.622154 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance
I20260808 18:26:34.622531 23842 external_mini_cluster.cc:1428] Setting key 2877a45c42f71d1788e30d6db09d4ad5
Aug 08 18:26:34 dist-test-slave-5k54 krb5kdc[9767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213594, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.131@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:26:34.630502 9857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:34.618662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:55091 (local address 127.23.72.190:41593)
0808 18:26:34.618814 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:34.618817 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:34.620107 (+ 1290us) server_negotiation.cc:407] Connection header received
0808 18:26:34.621004 (+ 897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:34.621007 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:34.621066 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:34.621195 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:34.622450 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:34.623037 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:34.624437 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:34.624703 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:34.624847 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:34.627378 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:34.627381 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:34.627383 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:26:34.627406 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:34.628715 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:34.629180 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:34.629182 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:34.629183 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:34.629277 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:34.629570 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:34.629572 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:34.629573 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:34.629749 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:34.629840 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:34.630280 (+ 440us) server_negotiation.cc:299] Negotiation successful
0808 18:26:34.630382 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 18:26:34.631335 10316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:34.619028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:41593 (local address 127.23.72.131:55091)
0808 18:26:34.619967 (+ 939us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:34.620009 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:34.620794 (+ 785us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:34.621363 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:34.621369 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:34.621732 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:26:34.622305 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:34.622321 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:34.623165 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:34.623169 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:34.624299 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:34.624312 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:34.624870 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:34.624873 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:34.625021 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:34.625614 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:26:34.625630 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:26:34.627207 (+ 1577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:26:34.628822 (+ 1615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:26:34.628827 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:26:34.628838 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:26:34.629089 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:26:34.629356 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:26:34.629358 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:26:34.629359 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:26:34.629486 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:26:34.629844 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:26:34.629849 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:26:34.630067 (+ 218us) client_negotiation.cc:769] Sending connection context
0808 18:26:34.630215 (+ 148us) client_negotiation.cc:240] Negotiation successful
0808 18:26:34.630454 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":748,"thread_start_us":609,"threads_started":1}
I20260808 18:26:34.632455 10314 heartbeater.cc:347] Connected to a master server at 127.23.72.190:41593
I20260808 18:26:34.632686 10314 heartbeater.cc:464] Registering TS with master...
I20260808 18:26:34.633162 10314 heartbeater.cc:511] Master 127.23.72.190:41593 requested a full tablet report, sending...
I20260808 18:26:34.634155 9816 ts_manager.cc:194] Registered new tserver with Master: 92188143c6d24314806938fd8a4269ed (127.23.72.131:37519)
I20260808 18:26:34.634716 9816 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.131@KRBTEST.COM'} at 127.23.72.131:55091
I20260808 18:26:34.637019 23842 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:26:34.646790 9857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:34.639053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42582 (local address 127.23.72.190:41593)
0808 18:26:34.639177 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:34.639201 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:34.639415 (+ 214us) server_negotiation.cc:407] Connection header received
0808 18:26:34.639622 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:34.639625 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:34.639662 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:34.639744 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:34.640440 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:34.640933 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:34.641783 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:34.642001 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:34.642107 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:34.643590 (+ 1483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:34.643593 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:34.643596 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:26:34.643623 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:34.645108 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:34.645537 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:34.645540 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:34.645542 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:34.645627 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:34.645908 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:34.645913 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:34.645915 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:34.646133 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:34.646272 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:34.646512 (+ 240us) server_negotiation.cc:299] Negotiation successful
0808 18:26:34.646626 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 18:26:34.649796 9816 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42582:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:26:34.653054 9816 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:26:34.668916 10330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:34.663660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32972 (local address 127.23.72.130:37291)
0808 18:26:34.664472 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:34.664481 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:34.664673 (+ 192us) server_negotiation.cc:407] Connection header received
0808 18:26:34.664843 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:34.664847 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:34.664972 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:34.665107 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:26:34.665348 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:34.666333 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:34.667628 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:34.668168 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:34.668289 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:34.668424 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:34.668648 (+ 224us) server_negotiation.cc:299] Negotiation successful
0808 18:26:34.668748 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":720,"thread_start_us":105,"threads_started":1}
I20260808 18:26:34.668910 10326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:34.663192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.130:37291 (local address 127.0.0.1:32972)
0808 18:26:34.664576 (+ 1384us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:34.664586 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:34.664699 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:34.665104 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:34.665107 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:34.665124 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:26:34.665222 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:34.665231 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:34.666489 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:34.666493 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:34.667520 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:34.667528 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:34.668292 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:34.668295 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:34.668434 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:34.668502 (+ 68us) client_negotiation.cc:769] Sending connection context
0808 18:26:34.668616 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 18:26:34.668748 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1337,"mutex_wait_us":29,"thread_start_us":123,"threads_started":1}
I20260808 18:26:34.669001 10328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:34.662589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39102 (local address 127.23.72.131:37519)
0808 18:26:34.663511 (+ 922us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:34.663527 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:34.663554 (+ 27us) server_negotiation.cc:407] Connection header received
0808 18:26:34.663633 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:34.663641 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:34.663897 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:34.664042 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:26:34.664393 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:34.665574 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:34.667195 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:34.668068 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:34.668176 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:34.668323 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:34.668727 (+ 404us) server_negotiation.cc:299] Negotiation successful
0808 18:26:34.668827 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":776,"thread_start_us":169,"threads_started":1}
I20260808 18:26:34.669176 10329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:34.662554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:35811 (local address 127.0.0.1:48536)
0808 18:26:34.663492 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:34.663506 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:34.663602 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:34.664924 (+ 1322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:34.664927 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:34.664944 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:26:34.665049 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:34.665055 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:34.666332 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:34.666335 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:34.667380 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:34.667391 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:34.668647 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:34.668650 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:34.668757 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:34.668777 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 18:26:34.668855 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 18:26:34.668916 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":843,"thread_start_us":109,"threads_started":1}
I20260808 18:26:34.669225 10331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:34.664030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48536 (local address 127.23.72.129:35811)
0808 18:26:34.664300 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:34.664313 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:34.664355 (+ 42us) server_negotiation.cc:407] Connection header received
0808 18:26:34.664434 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:34.664440 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:34.664617 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:34.664782 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:26:34.665178 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:34.666187 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:34.667563 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:34.668345 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:34.668461 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:34.668586 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:34.668990 (+ 404us) server_negotiation.cc:299] Negotiation successful
0808 18:26:34.669087 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":100,"threads_started":1}
I20260808 18:26:34.669464 10325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:34.662469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.131:37519 (local address 127.0.0.1:39102)
0808 18:26:34.662915 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:34.662963 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:34.663085 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:34.664074 (+ 989us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:34.664080 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:34.664105 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:26:34.664250 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:34.664257 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:34.665713 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:34.665717 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:34.667066 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:34.667076 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:34.668394 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:34.668401 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:34.668536 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:34.668558 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 18:26:34.668633 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 18:26:34.669318 (+ 685us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"mutex_wait_us":612,"thread_start_us":90,"threads_started":1}
I20260808 18:26:34.672312 10239 tablet_service.cc:1467] Processing CreateTablet for tablet c1324c97f7a04b75abbc8ce1e6197c12 (DEFAULT_TABLE table=test-table [id=ad7ed347b2a849938fe625438cb97050]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:26:34.672570 10094 tablet_service.cc:1467] Processing CreateTablet for tablet c1324c97f7a04b75abbc8ce1e6197c12 (DEFAULT_TABLE table=test-table [id=ad7ed347b2a849938fe625438cb97050]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:26:34.673457 10094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1324c97f7a04b75abbc8ce1e6197c12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:26:34.673470 10239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1324c97f7a04b75abbc8ce1e6197c12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:26:34.673286 9948 tablet_service.cc:1467] Processing CreateTablet for tablet c1324c97f7a04b75abbc8ce1e6197c12 (DEFAULT_TABLE table=test-table [id=ad7ed347b2a849938fe625438cb97050]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:26:34.674149 9948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1324c97f7a04b75abbc8ce1e6197c12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:26:34.679219 10332 tablet_bootstrap.cc:492] T c1324c97f7a04b75abbc8ce1e6197c12 P 865620b3ad1c4cd8a9b7b9806f9e1ede: Bootstrap starting.
I20260808 18:26:34.681324 10334 tablet_bootstrap.cc:492] T c1324c97f7a04b75abbc8ce1e6197c12 P 92188143c6d24314806938fd8a4269ed: Bootstrap starting.
I20260808 18:26:34.681478 10332 tablet_bootstrap.cc:654] T c1324c97f7a04b75abbc8ce1e6197c12 P 865620b3ad1c4cd8a9b7b9806f9e1ede: Neither blocks nor log segments found. Creating new log.
I20260808 18:26:34.682168 10333 tablet_bootstrap.cc:492] T c1324c97f7a04b75abbc8ce1e6197c12 P eb12b77914b0429f827ef7bfb0a0b010: Bootstrap starting.
I20260808 18:26:34.682615 10332 log.cc:824] T c1324c97f7a04b75abbc8ce1e6197c12 P 865620b3ad1c4cd8a9b7b9806f9e1ede: Log is configured to *not* fsync() on all Append() calls
I20260808 18:26:34.683521 10334 tablet_bootstrap.cc:654] T c1324c97f7a04b75abbc8ce1e6197c12 P 92188143c6d24314806938fd8a4269ed: Neither blocks nor log segments found. Creating new log.
I20260808 18:26:34.684389 10333 tablet_bootstrap.cc:654] T c1324c97f7a04b75abbc8ce1e6197c12 P eb12b77914b0429f827ef7bfb0a0b010: Neither blocks nor log segments found. Creating new log.
I20260808 18:26:34.684621 10332 tablet_bootstrap.cc:492] T c1324c97f7a04b75abbc8ce1e6197c12 P 865620b3ad1c4cd8a9b7b9806f9e1ede: No bootstrap required, opened a new log
I20260808 18:26:34.684824 10334 log.cc:824] T c1324c97f7a04b75abbc8ce1e6197c12 P 92188143c6d24314806938fd8a4269ed: Log is configured to *not* fsync() on all Append() calls
I20260808 18:26:34.684900 10332 ts_tablet_manager.cc:1397] T c1324c97f7a04b75abbc8ce1e6197c12 P 865620b3ad1c4cd8a9b7b9806f9e1ede: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 18:26:34.685547 10333 log.cc:824] T c1324c97f7a04b75abbc8ce1e6197c12 P eb12b77914b0429f827ef7bfb0a0b010: Log is configured to *not* fsync() on all Append() calls
I20260808 18:26:34.686787 10334 tablet_bootstrap.cc:492] T c1324c97f7a04b75abbc8ce1e6197c12 P 92188143c6d24314806938fd8a4269ed: No bootstrap required, opened a new log
I20260808 18:26:34.686980 10334 ts_tablet_manager.cc:1397] T c1324c97f7a04b75abbc8ce1e6197c12 P 92188143c6d24314806938fd8a4269ed: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260808 18:26:34.687268 10333 tablet_bootstrap.cc:492] T c1324c97f7a04b75abbc8ce1e6197c12 P eb12b77914b0429f827ef7bfb0a0b010: No bootstrap required, opened a new log
I20260808 18:26:34.687475 10333 ts_tablet_manager.cc:1397] T c1324c97f7a04b75abbc8ce1e6197c12 P eb12b77914b0429f827ef7bfb0a0b010: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260808 18:26:34.688854 10332 raft_consensus.cc:348] T c1324c97f7a04b75abbc8ce1e6197c12 P 865620b3ad1c4cd8a9b7b9806f9e1ede [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92188143c6d24314806938fd8a4269ed" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 37519 } } peers { permanent_uuid: "eb12b77914b0429f827ef7bfb0a0b010" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 35811 } } peers { permanent_uuid: "865620b3ad1c4cd8a9b7b9806f9e1ede" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 37291 } }
I20260808 18:26:34.689110 10332 raft_consensus.cc:374] T c1324c97f7a04b75abbc8ce1e6197c12 P 865620b3ad1c4cd8a9b7b9806f9e1ede [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:26:34.689193 10332 raft_consensus.cc:729] T c1324c97f7a04b75abbc8ce1e6197c12 P 865620b3ad1c4cd8a9b7b9806f9e1ede [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 865620b3ad1c4cd8a9b7b9806f9e1ede, State: Initialized, Role: FOLLOWER
I20260808 18:26:34.689644 10332 consensus_queue.cc:260] T c1324c97f7a04b75abbc8ce1e6197c12 P 865620b3ad1c4cd8a9b7b9806f9e1ede [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92188143c6d24314806938fd8a4269ed" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 37519 } } peers { permanent_uuid: "eb12b77914b0429f827ef7bfb0a0b010" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 35811 } } peers { permanent_uuid: "865620b3ad1c4cd8a9b7b9806f9e1ede" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 37291 } }
I20260808 18:26:34.690374 10164 heartbeater.cc:503] Master 127.23.72.190:41593 was elected leader, sending a full tablet report...
I20260808 18:26:34.690641 10332 ts_tablet_manager.cc:1428] T c1324c97f7a04b75abbc8ce1e6197c12 P 865620b3ad1c4cd8a9b7b9806f9e1ede: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 18:26:34.691252 10333 raft_consensus.cc:348] T c1324c97f7a04b75abbc8ce1e6197c12 P eb12b77914b0429f827ef7bfb0a0b010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92188143c6d24314806938fd8a4269ed" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 37519 } } peers { permanent_uuid: "eb12b77914b0429f827ef7bfb0a0b010" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 35811 } } peers { permanent_uuid: "865620b3ad1c4cd8a9b7b9806f9e1ede" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 37291 } }
I20260808 18:26:34.691552 10333 raft_consensus.cc:374] T c1324c97f7a04b75abbc8ce1e6197c12 P eb12b77914b0429f827ef7bfb0a0b010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:26:34.691665 10333 raft_consensus.cc:729] T c1324c97f7a04b75abbc8ce1e6197c12 P eb12b77914b0429f827ef7bfb0a0b010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb12b77914b0429f827ef7bfb0a0b010, State: Initialized, Role: FOLLOWER
I20260808 18:26:34.691520 10334 raft_consensus.cc:348] T c1324c97f7a04b75abbc8ce1e6197c12 P 92188143c6d24314806938fd8a4269ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92188143c6d24314806938fd8a4269ed" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 37519 } } peers { permanent_uuid: "eb12b77914b0429f827ef7bfb0a0b010" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 35811 } } peers { permanent_uuid: "865620b3ad1c4cd8a9b7b9806f9e1ede" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 37291 } }
I20260808 18:26:34.691815 10334 raft_consensus.cc:374] T c1324c97f7a04b75abbc8ce1e6197c12 P 92188143c6d24314806938fd8a4269ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:26:34.691903 10334 raft_consensus.cc:729] T c1324c97f7a04b75abbc8ce1e6197c12 P 92188143c6d24314806938fd8a4269ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92188143c6d24314806938fd8a4269ed, State: Initialized, Role: FOLLOWER
I20260808 18:26:34.692080 10333 consensus_queue.cc:260] T c1324c97f7a04b75abbc8ce1e6197c12 P eb12b77914b0429f827ef7bfb0a0b010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92188143c6d24314806938fd8a4269ed" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 37519 } } peers { permanent_uuid: "eb12b77914b0429f827ef7bfb0a0b010" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 35811 } } peers { permanent_uuid: "865620b3ad1c4cd8a9b7b9806f9e1ede" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 37291 } }
I20260808 18:26:34.692322 10334 consensus_queue.cc:260] T c1324c97f7a04b75abbc8ce1e6197c12 P 92188143c6d24314806938fd8a4269ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92188143c6d24314806938fd8a4269ed" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 37519 } } peers { permanent_uuid: "eb12b77914b0429f827ef7bfb0a0b010" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 35811 } } peers { permanent_uuid: "865620b3ad1c4cd8a9b7b9806f9e1ede" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 37291 } }
I20260808 18:26:34.692732 10018 heartbeater.cc:503] Master 127.23.72.190:41593 was elected leader, sending a full tablet report...
I20260808 18:26:34.693079 10333 ts_tablet_manager.cc:1428] T c1324c97f7a04b75abbc8ce1e6197c12 P eb12b77914b0429f827ef7bfb0a0b010: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 18:26:34.693285 10334 ts_tablet_manager.cc:1428] T c1324c97f7a04b75abbc8ce1e6197c12 P 92188143c6d24314806938fd8a4269ed: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 18:26:34.693611 10314 heartbeater.cc:503] Master 127.23.72.190:41593 was elected leader, sending a full tablet report...
I20260808 18:26:34.801242 10338 raft_consensus.cc:482] T c1324c97f7a04b75abbc8ce1e6197c12 P 865620b3ad1c4cd8a9b7b9806f9e1ede [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:26:34.801477 10338 raft_consensus.cc:504] T c1324c97f7a04b75abbc8ce1e6197c12 P 865620b3ad1c4cd8a9b7b9806f9e1ede [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92188143c6d24314806938fd8a4269ed" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 37519 } } peers { permanent_uuid: "eb12b77914b0429f827ef7bfb0a0b010" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 35811 } } peers { permanent_uuid: "865620b3ad1c4cd8a9b7b9806f9e1ede" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 37291 } }
I20260808 18:26:34.803015 10338 leader_election.cc:288] T c1324c97f7a04b75abbc8ce1e6197c12 P 865620b3ad1c4cd8a9b7b9806f9e1ede [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 92188143c6d24314806938fd8a4269ed (127.23.72.131:37519), eb12b77914b0429f827ef7bfb0a0b010 (127.23.72.129:35811)
I20260808 18:26:34.807090 10328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:34.803646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:36433 (local address 127.23.72.131:37519)
0808 18:26:34.803779 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:34.803783 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:34.803798 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:26:34.804007 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:34.804014 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:34.804061 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:34.804145 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:26:34.804432 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:34.805007 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:34.806102 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:34.806516 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:34.806648 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:34.806702 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:34.806892 (+ 190us) server_negotiation.cc:299] Negotiation successful
0808 18:26:34.806960 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 18:26:34.807090 10166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:34.803331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.131:37519 (local address 127.23.72.130:36433)
0808 18:26:34.803681 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:34.803697 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:34.803873 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:34.804165 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:34.804168 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:34.804189 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:26:34.804315 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:34.804321 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:34.805116 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:34.805122 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:34.805955 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:34.805974 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:34.806643 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:34.806646 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:34.806756 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:34.806779 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 18:26:34.806876 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 18:26:34.806954 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253}
I20260808 18:26:34.807240 10331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:34.803671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:59333 (local address 127.23.72.129:35811)
0808 18:26:34.803815 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:34.803820 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:34.804030 (+ 210us) server_negotiation.cc:407] Connection header received
0808 18:26:34.804194 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:34.804203 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:34.804246 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:34.804335 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:26:34.804629 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:34.805159 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:34.806234 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:34.806651 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:34.806813 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:34.806874 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:34.807044 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 18:26:34.807115 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 18:26:34.807261 10341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:34.803331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:35811 (local address 127.23.72.130:59333)
0808 18:26:34.803919 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:34.803930 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:34.804065 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:34.804346 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:34.804349 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:34.804372 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:26:34.804508 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:34.804514 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:34.805282 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:34.805286 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:34.806102 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:34.806111 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:34.806805 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:34.806808 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:34.806910 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:34.806934 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 18:26:34.807026 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 18:26:34.807109 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"mutex_wait_us":180,"thread_start_us":138,"threads_started":1}
I20260808 18:26:34.807880 10261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1324c97f7a04b75abbc8ce1e6197c12" candidate_uuid: "865620b3ad1c4cd8a9b7b9806f9e1ede" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92188143c6d24314806938fd8a4269ed" is_pre_election: true
I20260808 18:26:34.807879 9969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1324c97f7a04b75abbc8ce1e6197c12" candidate_uuid: "865620b3ad1c4cd8a9b7b9806f9e1ede" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb12b77914b0429f827ef7bfb0a0b010" is_pre_election: true
I20260808 18:26:34.808213 9969 raft_consensus.cc:2454] T c1324c97f7a04b75abbc8ce1e6197c12 P eb12b77914b0429f827ef7bfb0a0b010 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 865620b3ad1c4cd8a9b7b9806f9e1ede in term 0.
I20260808 18:26:34.808213 10261 raft_consensus.cc:2454] T c1324c97f7a04b75abbc8ce1e6197c12 P 92188143c6d24314806938fd8a4269ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 865620b3ad1c4cd8a9b7b9806f9e1ede in term 0.
I20260808 18:26:34.808799 10046 leader_election.cc:302] T c1324c97f7a04b75abbc8ce1e6197c12 P 865620b3ad1c4cd8a9b7b9806f9e1ede [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 865620b3ad1c4cd8a9b7b9806f9e1ede, 92188143c6d24314806938fd8a4269ed, eb12b77914b0429f827ef7bfb0a0b010; no voters:
I20260808 18:26:34.809094 10338 raft_consensus.cc:2781] T c1324c97f7a04b75abbc8ce1e6197c12 P 865620b3ad1c4cd8a9b7b9806f9e1ede [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:26:34.809180 10338 raft_consensus.cc:482] T c1324c97f7a04b75abbc8ce1e6197c12 P 865620b3ad1c4cd8a9b7b9806f9e1ede [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:26:34.809226 10338 raft_consensus.cc:3037] T c1324c97f7a04b75abbc8ce1e6197c12 P 865620b3ad1c4cd8a9b7b9806f9e1ede [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:26:34.810312 10338 raft_consensus.cc:504] T c1324c97f7a04b75abbc8ce1e6197c12 P 865620b3ad1c4cd8a9b7b9806f9e1ede [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92188143c6d24314806938fd8a4269ed" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 37519 } } peers { permanent_uuid: "eb12b77914b0429f827ef7bfb0a0b010" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 35811 } } peers { permanent_uuid: "865620b3ad1c4cd8a9b7b9806f9e1ede" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 37291 } }
I20260808 18:26:34.810648 10338 leader_election.cc:288] T c1324c97f7a04b75abbc8ce1e6197c12 P 865620b3ad1c4cd8a9b7b9806f9e1ede [CANDIDATE]: Term 1 election: Requested vote from peers 92188143c6d24314806938fd8a4269ed (127.23.72.131:37519), eb12b77914b0429f827ef7bfb0a0b010 (127.23.72.129:35811)
I20260808 18:26:34.810977 10261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1324c97f7a04b75abbc8ce1e6197c12" candidate_uuid: "865620b3ad1c4cd8a9b7b9806f9e1ede" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92188143c6d24314806938fd8a4269ed"
I20260808 18:26:34.811000 9969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1324c97f7a04b75abbc8ce1e6197c12" candidate_uuid: "865620b3ad1c4cd8a9b7b9806f9e1ede" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb12b77914b0429f827ef7bfb0a0b010"
I20260808 18:26:34.811115 9969 raft_consensus.cc:3037] T c1324c97f7a04b75abbc8ce1e6197c12 P eb12b77914b0429f827ef7bfb0a0b010 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:26:34.811115 10261 raft_consensus.cc:3037] T c1324c97f7a04b75abbc8ce1e6197c12 P 92188143c6d24314806938fd8a4269ed [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:26:34.812475 9969 raft_consensus.cc:2454] T c1324c97f7a04b75abbc8ce1e6197c12 P eb12b77914b0429f827ef7bfb0a0b010 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 865620b3ad1c4cd8a9b7b9806f9e1ede in term 1.
I20260808 18:26:34.812475 10261 raft_consensus.cc:2454] T c1324c97f7a04b75abbc8ce1e6197c12 P 92188143c6d24314806938fd8a4269ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 865620b3ad1c4cd8a9b7b9806f9e1ede in term 1.
I20260808 18:26:34.812825 10045 leader_election.cc:302] T c1324c97f7a04b75abbc8ce1e6197c12 P 865620b3ad1c4cd8a9b7b9806f9e1ede [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 865620b3ad1c4cd8a9b7b9806f9e1ede, 92188143c6d24314806938fd8a4269ed; no voters:
I20260808 18:26:34.813057 10338 raft_consensus.cc:2781] T c1324c97f7a04b75abbc8ce1e6197c12 P 865620b3ad1c4cd8a9b7b9806f9e1ede [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:26:34.813244 10338 raft_consensus.cc:686] T c1324c97f7a04b75abbc8ce1e6197c12 P 865620b3ad1c4cd8a9b7b9806f9e1ede [term 1 LEADER]: Becoming Leader. State: Replica: 865620b3ad1c4cd8a9b7b9806f9e1ede, State: Running, Role: LEADER
I20260808 18:26:34.813589 10338 consensus_queue.cc:237] T c1324c97f7a04b75abbc8ce1e6197c12 P 865620b3ad1c4cd8a9b7b9806f9e1ede [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92188143c6d24314806938fd8a4269ed" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 37519 } } peers { permanent_uuid: "eb12b77914b0429f827ef7bfb0a0b010" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 35811 } } peers { permanent_uuid: "865620b3ad1c4cd8a9b7b9806f9e1ede" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 37291 } }
I20260808 18:26:34.817001 9816 catalog_manager.cc:5495] T c1324c97f7a04b75abbc8ce1e6197c12 P 865620b3ad1c4cd8a9b7b9806f9e1ede reported cstate change: term changed from 0 to 1, leader changed from <none> to 865620b3ad1c4cd8a9b7b9806f9e1ede (127.23.72.130). New cstate: current_term: 1 leader_uuid: "865620b3ad1c4cd8a9b7b9806f9e1ede" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92188143c6d24314806938fd8a4269ed" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 37519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb12b77914b0429f827ef7bfb0a0b010" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 35811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "865620b3ad1c4cd8a9b7b9806f9e1ede" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 37291 } health_report { overall_health: HEALTHY } } }
I20260808 18:26:34.864516 10330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:34.860696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32980 (local address 127.23.72.130:37291)
0808 18:26:34.860860 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:34.860863 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:34.860878 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:26:34.860981 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:34.860984 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:34.861045 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:34.861141 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:26:34.861469 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:34.862168 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:34.863088 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:34.863317 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:34.863401 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:34.863752 (+ 351us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:26:34.864174 (+ 422us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:26:34.864274 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:34.864340 (+ 66us) server_negotiation.cc:299] Negotiation successful
0808 18:26:34.864390 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260808 18:26:34.867709 10315 tablet.cc:2369] T c1324c97f7a04b75abbc8ce1e6197c12 P 92188143c6d24314806938fd8a4269ed: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:26:34.870790 9969 raft_consensus.cc:1261] T c1324c97f7a04b75abbc8ce1e6197c12 P eb12b77914b0429f827ef7bfb0a0b010 [term 1 FOLLOWER]: Refusing update from remote peer 865620b3ad1c4cd8a9b7b9806f9e1ede: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:26:34.870802 10261 raft_consensus.cc:1261] T c1324c97f7a04b75abbc8ce1e6197c12 P 92188143c6d24314806938fd8a4269ed [term 1 FOLLOWER]: Refusing update from remote peer 865620b3ad1c4cd8a9b7b9806f9e1ede: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:26:34.871599 10342 consensus_queue.cc:1035] T c1324c97f7a04b75abbc8ce1e6197c12 P 865620b3ad1c4cd8a9b7b9806f9e1ede [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb12b77914b0429f827ef7bfb0a0b010" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 35811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:26:34.871815 10338 consensus_queue.cc:1035] T c1324c97f7a04b75abbc8ce1e6197c12 P 865620b3ad1c4cd8a9b7b9806f9e1ede [LEADER]: Connected to new peer: Peer: permanent_uuid: "92188143c6d24314806938fd8a4269ed" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 37519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:26:34.879439 10347 mvcc.cc:205] Tried to move back new op lower bound from 7316330884582150144 to 7316330884359397376. Current Snapshot: MvccSnapshot[applied={T|T < 7316330884582150144}]
I20260808 18:26:34.888603 9815 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42582:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:26:34.888760 9815 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42582:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:26:34.891223 9815 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1f3ab6be0e4c4d6294fe32fa8e3f1ada: Sending DeleteTablet for 3 replicas of tablet c1324c97f7a04b75abbc8ce1e6197c12
I20260808 18:26:34.892060 10239 tablet_service.cc:1514] Processing DeleteTablet for tablet c1324c97f7a04b75abbc8ce1e6197c12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:26:34 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:39102
I20260808 18:26:34.892205 9948 tablet_service.cc:1514] Processing DeleteTablet for tablet c1324c97f7a04b75abbc8ce1e6197c12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:26:34 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:48536
I20260808 18:26:34.892310 10094 tablet_service.cc:1514] Processing DeleteTablet for tablet c1324c97f7a04b75abbc8ce1e6197c12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:26:34 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:32972
I20260808 18:26:34.892642 10356 tablet_replica.cc:330] T c1324c97f7a04b75abbc8ce1e6197c12 P eb12b77914b0429f827ef7bfb0a0b010: stopping tablet replica
I20260808 18:26:34.892745 10355 tablet_replica.cc:330] T c1324c97f7a04b75abbc8ce1e6197c12 P 92188143c6d24314806938fd8a4269ed: stopping tablet replica
I20260808 18:26:34.893054 10355 raft_consensus.cc:2229] T c1324c97f7a04b75abbc8ce1e6197c12 P 92188143c6d24314806938fd8a4269ed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:26:34.893042 10356 raft_consensus.cc:2229] T c1324c97f7a04b75abbc8ce1e6197c12 P eb12b77914b0429f827ef7bfb0a0b010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:26:34.893374 10355 raft_consensus.cc:2258] T c1324c97f7a04b75abbc8ce1e6197c12 P 92188143c6d24314806938fd8a4269ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:26:34.893374 10356 raft_consensus.cc:2258] T c1324c97f7a04b75abbc8ce1e6197c12 P eb12b77914b0429f827ef7bfb0a0b010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:26:34.893545 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 9880
I20260808 18:26:34.896188 10357 tablet_replica.cc:330] T c1324c97f7a04b75abbc8ce1e6197c12 P 865620b3ad1c4cd8a9b7b9806f9e1ede: stopping tablet replica
I20260808 18:26:34.896504 10357 raft_consensus.cc:2229] T c1324c97f7a04b75abbc8ce1e6197c12 P 865620b3ad1c4cd8a9b7b9806f9e1ede [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:26:34.896914 10357 raft_consensus.cc:2258] T c1324c97f7a04b75abbc8ce1e6197c12 P 865620b3ad1c4cd8a9b7b9806f9e1ede [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:26:34.897091 10355 ts_tablet_manager.cc:1905] T c1324c97f7a04b75abbc8ce1e6197c12 P 92188143c6d24314806938fd8a4269ed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:26:34.898173 10357 ts_tablet_manager.cc:1905] T c1324c97f7a04b75abbc8ce1e6197c12 P 865620b3ad1c4cd8a9b7b9806f9e1ede: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:26:34.899935 10355 ts_tablet_manager.cc:1918] T c1324c97f7a04b75abbc8ce1e6197c12 P 92188143c6d24314806938fd8a4269ed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:26:34.900113 10355 log.cc:1196] T c1324c97f7a04b75abbc8ce1e6197c12 P 92188143c6d24314806938fd8a4269ed: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-2/wal/wals/c1324c97f7a04b75abbc8ce1e6197c12
I20260808 18:26:34.900545 10355 ts_tablet_manager.cc:1939] T c1324c97f7a04b75abbc8ce1e6197c12 P 92188143c6d24314806938fd8a4269ed: Deleting consensus metadata
I20260808 18:26:34.901072 10357 ts_tablet_manager.cc:1918] T c1324c97f7a04b75abbc8ce1e6197c12 P 865620b3ad1c4cd8a9b7b9806f9e1ede: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:26:34.901223 10357 log.cc:1196] T c1324c97f7a04b75abbc8ce1e6197c12 P 865620b3ad1c4cd8a9b7b9806f9e1ede: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213345431840-23842-0/minicluster-data/ts-1/wal/wals/c1324c97f7a04b75abbc8ce1e6197c12
I20260808 18:26:34.901597 10357 ts_tablet_manager.cc:1939] T c1324c97f7a04b75abbc8ce1e6197c12 P 865620b3ad1c4cd8a9b7b9806f9e1ede: Deleting consensus metadata
I20260808 18:26:34.901948 9799 catalog_manager.cc:4841] TS 92188143c6d24314806938fd8a4269ed (127.23.72.131:37519): tablet c1324c97f7a04b75abbc8ce1e6197c12 (table test-table [id=ad7ed347b2a849938fe625438cb97050]) successfully deleted
I20260808 18:26:34.902704 9800 catalog_manager.cc:4841] TS 865620b3ad1c4cd8a9b7b9806f9e1ede (127.23.72.130:37291): tablet c1324c97f7a04b75abbc8ce1e6197c12 (table test-table [id=ad7ed347b2a849938fe625438cb97050]) successfully deleted
W20260808 18:26:34.905649 9800 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.72.129:35811 (error 108)
I20260808 18:26:34.905781 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 10025
I20260808 18:26:34.907943 10325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:34.907573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:35811 (local address 127.0.0.1:48538)
0808 18:26:34.907771 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:34.907846 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.129:35811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
W20260808 18:26:34.908205 9800 catalog_manager.cc:4584] TS eb12b77914b0429f827ef7bfb0a0b010 (127.23.72.129:35811): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c1324c97f7a04b75abbc8ce1e6197c12: Network error: Client connection negotiation failed: client connection to 127.23.72.129:35811: connect: Connection refused (error 111)
I20260808 18:26:34.913100 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 10169
I20260808 18:26:34.920009 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 9780
2026-08-08T18:26:34Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2494 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-08T18:26:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:26:34Z Disabled control of system clock
I20260808 18:26:34.953723 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:45781
--webserver_interface=127.23.72.190
--webserver_port=0
--builtin_ntp_servers=127.23.72.148:44117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:45781
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 18:26:35.082229 10362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:26:35.082495 10362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:26:35.082531 10362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:26:35.086548 10362 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:26:35.086602 10362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:26:35.086618 10362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:26:35.086634 10362 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:26:35.086676 10362 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:26:35.091351 10362 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:44117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:45781
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:45781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.72.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:26:35.092415 10362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:26:35.093775 10362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:26:35.102357 10367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:35.102365 10370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:35.102375 10368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:35.102561 10362 server_base.cc:1034] running on GCE node
I20260808 18:26:35.103364 10362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:26:35.104478 10362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:26:35.105638 10362 hybrid_clock.cc:648] HybridClock initialized: now 1786213595105614 us; error 35 us; skew 500 ppm
I20260808 18:26:35.107563 10362 webserver.cc:460] Webserver started at http://127.23.72.190:45987/ using document root <none> and password file <none>
I20260808 18:26:35.108147 10362 fs_manager.cc:359] Metadata directory not provided
I20260808 18:26:35.108208 10362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:26:35.108445 10362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:26:35.110502 10362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "d2a3c733c5ce42a49024e2ebe0f6b393"
format_stamp: "Formatted at 2026-08-08 18:26:35 on dist-test-slave-5k54"
server_key: "0aa374cb12c5cece392391c52128dc29"
server_key_iv: "f1656d8aa994e891ebd68a56b2ec862b"
server_key_version: "encryptionkey@0"
I20260808 18:26:35.111080 10362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "d2a3c733c5ce42a49024e2ebe0f6b393"
format_stamp: "Formatted at 2026-08-08 18:26:35 on dist-test-slave-5k54"
server_key: "0aa374cb12c5cece392391c52128dc29"
server_key_iv: "f1656d8aa994e891ebd68a56b2ec862b"
server_key_version: "encryptionkey@0"
I20260808 18:26:35.114810 10362 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260808 18:26:35.117347 10376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:35.118676 10362 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:26:35.118860 10362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "d2a3c733c5ce42a49024e2ebe0f6b393"
format_stamp: "Formatted at 2026-08-08 18:26:35 on dist-test-slave-5k54"
server_key: "0aa374cb12c5cece392391c52128dc29"
server_key_iv: "f1656d8aa994e891ebd68a56b2ec862b"
server_key_version: "encryptionkey@0"
I20260808 18:26:35.118973 10362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:26:35.138573 10362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:26:35.139268 10362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:26:35.139521 10362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:26:35.146740 10362 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.190:45781
I20260808 18:26:35.146756 10432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.190:45781 every 8 connection(s)
I20260808 18:26:35.147713 10362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:26:35.149845 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 10362
I20260808 18:26:35.150008 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:26:35.150296 23842 external_mini_cluster.cc:1428] Setting key 20895ee138efe4e41309bbef0b02f603
I20260808 18:26:35.150985 10433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:26:35.158092 10433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2a3c733c5ce42a49024e2ebe0f6b393: Bootstrap starting.
I20260808 18:26:35.160821 10439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:35.151774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48436 (local address 127.23.72.190:45781)
0808 18:26:35.152468 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:35.152488 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:35.152532 (+ 44us) server_negotiation.cc:407] Connection header received
0808 18:26:35.153297 (+ 765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:35.153319 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:35.153830 (+ 511us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:35.154199 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:35.154648 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:35.156128 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:35.157039 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:35.157733 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:35.157888 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:35.158728 (+ 840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:35.158734 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:35.158748 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:26:35.159224 (+ 476us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:35.159299 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:26:35.159330 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:35.159469 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:35.159625 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 18:26:35.159879 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":136,"threads_started":1}
I20260808 18:26:35.160817 10433 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2a3c733c5ce42a49024e2ebe0f6b393: Neither blocks nor log segments found. Creating new log.
I20260808 18:26:35.161643 10433 log.cc:824] T 00000000000000000000000000000000 P d2a3c733c5ce42a49024e2ebe0f6b393: Log is configured to *not* fsync() on all Append() calls
I20260808 18:26:35.163681 10433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2a3c733c5ce42a49024e2ebe0f6b393: No bootstrap required, opened a new log
I20260808 18:26:35.167371 10433 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2a3c733c5ce42a49024e2ebe0f6b393 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2a3c733c5ce42a49024e2ebe0f6b393" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 45781 } }
I20260808 18:26:35.167642 10433 raft_consensus.cc:374] T 00000000000000000000000000000000 P d2a3c733c5ce42a49024e2ebe0f6b393 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:26:35.167721 10433 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2a3c733c5ce42a49024e2ebe0f6b393 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2a3c733c5ce42a49024e2ebe0f6b393, State: Initialized, Role: FOLLOWER
I20260808 18:26:35.168125 10433 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2a3c733c5ce42a49024e2ebe0f6b393 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2a3c733c5ce42a49024e2ebe0f6b393" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 45781 } }
I20260808 18:26:35.168265 10433 raft_consensus.cc:388] T 00000000000000000000000000000000 P d2a3c733c5ce42a49024e2ebe0f6b393 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:26:35.168347 10433 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2a3c733c5ce42a49024e2ebe0f6b393 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:26:35.168463 10433 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2a3c733c5ce42a49024e2ebe0f6b393 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:26:35.169348 10433 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2a3c733c5ce42a49024e2ebe0f6b393 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2a3c733c5ce42a49024e2ebe0f6b393" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 45781 } }
I20260808 18:26:35.169680 10433 leader_election.cc:302] T 00000000000000000000000000000000 P d2a3c733c5ce42a49024e2ebe0f6b393 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d2a3c733c5ce42a49024e2ebe0f6b393; no voters:
I20260808 18:26:35.169979 10433 leader_election.cc:288] T 00000000000000000000000000000000 P d2a3c733c5ce42a49024e2ebe0f6b393 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:26:35.170123 10441 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d2a3c733c5ce42a49024e2ebe0f6b393 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:26:35.170403 10441 raft_consensus.cc:686] T 00000000000000000000000000000000 P d2a3c733c5ce42a49024e2ebe0f6b393 [term 1 LEADER]: Becoming Leader. State: Replica: d2a3c733c5ce42a49024e2ebe0f6b393, State: Running, Role: LEADER
I20260808 18:26:35.170699 10441 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2a3c733c5ce42a49024e2ebe0f6b393 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2a3c733c5ce42a49024e2ebe0f6b393" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 45781 } }
I20260808 18:26:35.171269 10433 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2a3c733c5ce42a49024e2ebe0f6b393 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:26:35.172394 10443 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2a3c733c5ce42a49024e2ebe0f6b393 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2a3c733c5ce42a49024e2ebe0f6b393. Latest consensus state: current_term: 1 leader_uuid: "d2a3c733c5ce42a49024e2ebe0f6b393" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2a3c733c5ce42a49024e2ebe0f6b393" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 45781 } } }
I20260808 18:26:35.172539 10443 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2a3c733c5ce42a49024e2ebe0f6b393 [sys.catalog]: This master's current role is: LEADER
I20260808 18:26:35.172712 10442 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2a3c733c5ce42a49024e2ebe0f6b393 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d2a3c733c5ce42a49024e2ebe0f6b393" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2a3c733c5ce42a49024e2ebe0f6b393" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 45781 } } }
I20260808 18:26:35.172807 10442 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2a3c733c5ce42a49024e2ebe0f6b393 [sys.catalog]: This master's current role is: LEADER
I20260808 18:26:35.173840 10454 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:26:35.176556 10454 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:26:35.181391 10454 catalog_manager.cc:1347] Generated new cluster ID: 921974b4465b473aa1c268462c67e27b
I20260808 18:26:35.181453 10454 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:26:35.200955 10454 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:26:35.202229 10454 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:26:35.210830 10454 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d2a3c733c5ce42a49024e2ebe0f6b393: Generated new TSK 0
I20260808 18:26:35.211584 10454 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:26:35.214905 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.129:0
--local_ip_for_outbound_sockets=127.23.72.129
--webserver_interface=127.23.72.129
--webserver_port=0
--tserver_master_addrs=127.23.72.190:45781
--builtin_ntp_servers=127.23.72.148:44117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 18:26:35.328967 10461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:26:35.329213 10461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:26:35.329252 10461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:26:35.333321 10461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:26:35.333371 10461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:26:35.333423 10461 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.129
I20260808 18:26:35.338040 10461 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:44117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.72.129
--webserver_port=0
--tserver_master_addrs=127.23.72.190:45781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.129
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:26:35.339054 10461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:26:35.340454 10461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:26:35.350085 10469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:35.350137 10467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:35.350114 10461 server_base.cc:1034] running on GCE node
W20260808 18:26:35.350085 10466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:35.350764 10461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:26:35.351310 10461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:26:35.352473 10461 hybrid_clock.cc:648] HybridClock initialized: now 1786213595352452 us; error 32 us; skew 500 ppm
I20260808 18:26:35.354385 10461 webserver.cc:460] Webserver started at http://127.23.72.129:44129/ using document root <none> and password file <none>
I20260808 18:26:35.354954 10461 fs_manager.cc:359] Metadata directory not provided
I20260808 18:26:35.355015 10461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:26:35.355261 10461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:26:35.357292 10461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-0/data/instance:
uuid: "124426d2d97049e786f93096f8ca3957"
format_stamp: "Formatted at 2026-08-08 18:26:35 on dist-test-slave-5k54"
server_key: "e67d78d213ae5a0f40f5b049a1294c7c"
server_key_iv: "312ae51df2ad6b8e1b5c90af634c25b7"
server_key_version: "encryptionkey@0"
I20260808 18:26:35.357837 10461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance:
uuid: "124426d2d97049e786f93096f8ca3957"
format_stamp: "Formatted at 2026-08-08 18:26:35 on dist-test-slave-5k54"
server_key: "e67d78d213ae5a0f40f5b049a1294c7c"
server_key_iv: "312ae51df2ad6b8e1b5c90af634c25b7"
server_key_version: "encryptionkey@0"
I20260808 18:26:35.361447 10461 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260808 18:26:35.363821 10476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:35.364991 10461 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 18:26:35.365114 10461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-0/wal
uuid: "124426d2d97049e786f93096f8ca3957"
format_stamp: "Formatted at 2026-08-08 18:26:35 on dist-test-slave-5k54"
server_key: "e67d78d213ae5a0f40f5b049a1294c7c"
server_key_iv: "312ae51df2ad6b8e1b5c90af634c25b7"
server_key_version: "encryptionkey@0"
I20260808 18:26:35.365213 10461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:26:35.376958 10461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:26:35.377588 10461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:26:35.377780 10461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:26:35.378361 10461 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:26:35.379279 10461 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:26:35.379325 10461 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:35.379391 10461 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:26:35.379437 10461 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:35.388219 10461 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.129:44435
I20260808 18:26:35.388342 10594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.129:44435 every 8 connection(s)
I20260808 18:26:35.389149 10461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
I20260808 18:26:35.392048 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 10461
I20260808 18:26:35.392167 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance
I20260808 18:26:35.392504 23842 external_mini_cluster.cc:1428] Setting key cc5752f8398470256adf9a638b036656
I20260808 18:26:35.394750 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.130:0
--local_ip_for_outbound_sockets=127.23.72.130
--webserver_interface=127.23.72.130
--webserver_port=0
--tserver_master_addrs=127.23.72.190:45781
--builtin_ntp_servers=127.23.72.148:44117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 18:26:35.399471 10439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:35.390996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:43331 (local address 127.23.72.190:45781)
0808 18:26:35.391171 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:35.391174 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:35.392480 (+ 1306us) server_negotiation.cc:407] Connection header received
0808 18:26:35.394117 (+ 1637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:35.394123 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:35.394182 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:35.394250 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:35.395486 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:35.396006 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:35.397351 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:35.397617 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:35.397738 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:35.398827 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:35.398834 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:35.398850 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:26:35.398869 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:35.398896 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:26:35.398911 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:35.399006 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:35.399148 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 18:26:35.399244 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 18:26:35.400279 10597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:35.391469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:45781 (local address 127.23.72.129:43331)
0808 18:26:35.392283 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:35.392358 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:35.393918 (+ 1560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:35.394459 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:35.394468 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:35.394581 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:26:35.395268 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:35.395289 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:35.396135 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:35.396142 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:35.397214 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:35.397223 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:35.397790 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:35.397793 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:35.397948 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:35.398569 (+ 621us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:26:35.398570 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:26:35.398609 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:26:35.398668 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:26:35.399012 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:26:35.399020 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:26:35.399023 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 18:26:35.399112 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 18:26:35.399340 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":108,"threads_started":1}
I20260808 18:26:35.401513 10595 heartbeater.cc:347] Connected to a master server at 127.23.72.190:45781
I20260808 18:26:35.401773 10595 heartbeater.cc:464] Registering TS with master...
I20260808 18:26:35.402410 10595 heartbeater.cc:511] Master 127.23.72.190:45781 requested a full tablet report, sending...
I20260808 18:26:35.404430 10395 ts_manager.cc:194] Registered new tserver with Master: 124426d2d97049e786f93096f8ca3957 (127.23.72.129:44435)
I20260808 18:26:35.406157 10395 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.72.129:43331
W20260808 18:26:35.509197 10598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:26:35.509444 10598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:26:35.509482 10598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:26:35.513408 10598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:26:35.513459 10598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:26:35.513512 10598 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.130
I20260808 18:26:35.518198 10598 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:44117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.72.130
--webserver_port=0
--tserver_master_addrs=127.23.72.190:45781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10598
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.130
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:26:35.519166 10598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:26:35.520560 10598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:26:35.530150 10606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:35.530166 10603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:35.530155 10604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:35.530515 10598 server_base.cc:1034] running on GCE node
I20260808 18:26:35.530910 10598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:26:35.531597 10598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:26:35.532814 10598 hybrid_clock.cc:648] HybridClock initialized: now 1786213595532792 us; error 33 us; skew 500 ppm
I20260808 18:26:35.534621 10598 webserver.cc:460] Webserver started at http://127.23.72.130:35223/ using document root <none> and password file <none>
I20260808 18:26:35.535197 10598 fs_manager.cc:359] Metadata directory not provided
I20260808 18:26:35.535288 10598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:26:35.535676 10598 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:26:35.537678 10598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-1/data/instance:
uuid: "bbc22d236cfe42cf81727c9f6c826b63"
format_stamp: "Formatted at 2026-08-08 18:26:35 on dist-test-slave-5k54"
server_key: "a7d014fc69387d51b32412833343db99"
server_key_iv: "82343125022cd007e4c92593bbccf6ff"
server_key_version: "encryptionkey@0"
I20260808 18:26:35.538239 10598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance:
uuid: "bbc22d236cfe42cf81727c9f6c826b63"
format_stamp: "Formatted at 2026-08-08 18:26:35 on dist-test-slave-5k54"
server_key: "a7d014fc69387d51b32412833343db99"
server_key_iv: "82343125022cd007e4c92593bbccf6ff"
server_key_version: "encryptionkey@0"
I20260808 18:26:35.541803 10598 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 18:26:35.544282 10614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:35.545527 10598 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:26:35.545642 10598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-1/wal
uuid: "bbc22d236cfe42cf81727c9f6c826b63"
format_stamp: "Formatted at 2026-08-08 18:26:35 on dist-test-slave-5k54"
server_key: "a7d014fc69387d51b32412833343db99"
server_key_iv: "82343125022cd007e4c92593bbccf6ff"
server_key_version: "encryptionkey@0"
I20260808 18:26:35.545743 10598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:26:35.568661 10598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:26:35.569432 10598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:26:35.569641 10598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:26:35.570227 10598 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:26:35.571152 10598 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:26:35.571200 10598 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:35.571240 10598 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:26:35.571264 10598 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:35.580551 10598 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.130:44991
I20260808 18:26:35.580581 10732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.130:44991 every 8 connection(s)
I20260808 18:26:35.581462 10598 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
I20260808 18:26:35.582646 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 10598
I20260808 18:26:35.582787 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance
I20260808 18:26:35.583117 23842 external_mini_cluster.cc:1428] Setting key 8dfa3ed64312577b990e38a91969f1b3
I20260808 18:26:35.585211 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.131:0
--local_ip_for_outbound_sockets=127.23.72.131
--webserver_interface=127.23.72.131
--webserver_port=0
--tserver_master_addrs=127.23.72.190:45781
--builtin_ntp_servers=127.23.72.148:44117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 18:26:35.591627 10439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:35.583297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:42889 (local address 127.23.72.190:45781)
0808 18:26:35.583493 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:35.583498 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:35.584807 (+ 1309us) server_negotiation.cc:407] Connection header received
0808 18:26:35.586248 (+ 1441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:35.586254 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:35.586319 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:35.586414 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:35.587278 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:35.588148 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:35.589270 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:35.589710 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:35.589838 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:35.590997 (+ 1159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:35.591006 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:35.591011 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:26:35.591027 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:35.591056 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:26:35.591071 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:35.591158 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:35.591314 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 18:26:35.591393 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 18:26:35.592399 10735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:35.583834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:45781 (local address 127.23.72.130:42889)
0808 18:26:35.584595 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:35.584648 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:35.586043 (+ 1395us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:35.586572 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:35.586578 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:35.586658 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:26:35.587129 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:35.587141 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:35.588275 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:35.588277 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:35.589137 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:35.589145 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:35.589903 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:35.589906 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:35.590057 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:35.590719 (+ 662us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:26:35.590721 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:26:35.590759 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:26:35.590820 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:26:35.591164 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:26:35.591171 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:26:35.591175 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 18:26:35.591270 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 18:26:35.591545 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":146,"threads_started":1}
I20260808 18:26:35.593571 10733 heartbeater.cc:347] Connected to a master server at 127.23.72.190:45781
I20260808 18:26:35.593829 10733 heartbeater.cc:464] Registering TS with master...
I20260808 18:26:35.594501 10733 heartbeater.cc:511] Master 127.23.72.190:45781 requested a full tablet report, sending...
I20260808 18:26:35.595690 10395 ts_manager.cc:194] Registered new tserver with Master: bbc22d236cfe42cf81727c9f6c826b63 (127.23.72.130:44991)
I20260808 18:26:35.596261 10395 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.72.130:42889
W20260808 18:26:35.701313 10736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:26:35.701546 10736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:26:35.701582 10736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:26:35.705567 10736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:26:35.705616 10736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:26:35.705698 10736 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.131
I20260808 18:26:35.710439 10736 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:44117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.72.131
--webserver_port=0
--tserver_master_addrs=127.23.72.190:45781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.131
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:26:35.711546 10736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:26:35.712913 10736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:26:35.722205 10742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:35.722215 10744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:35.722203 10741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:35.722800 10736 server_base.cc:1034] running on GCE node
I20260808 18:26:35.723217 10736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:26:35.723893 10736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:26:35.725071 10736 hybrid_clock.cc:648] HybridClock initialized: now 1786213595725038 us; error 41 us; skew 500 ppm
I20260808 18:26:35.726950 10736 webserver.cc:460] Webserver started at http://127.23.72.131:38245/ using document root <none> and password file <none>
I20260808 18:26:35.727568 10736 fs_manager.cc:359] Metadata directory not provided
I20260808 18:26:35.727636 10736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:26:35.727841 10736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:26:35.729831 10736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-2/data/instance:
uuid: "dcc484db0c6545c4b9b7cc2d2689776c"
format_stamp: "Formatted at 2026-08-08 18:26:35 on dist-test-slave-5k54"
server_key: "e3156f8da6e6c16d1bf739e195be5887"
server_key_iv: "ef3d6a5737c95bc6ec9af61626450e28"
server_key_version: "encryptionkey@0"
I20260808 18:26:35.730382 10736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance:
uuid: "dcc484db0c6545c4b9b7cc2d2689776c"
format_stamp: "Formatted at 2026-08-08 18:26:35 on dist-test-slave-5k54"
server_key: "e3156f8da6e6c16d1bf739e195be5887"
server_key_iv: "ef3d6a5737c95bc6ec9af61626450e28"
server_key_version: "encryptionkey@0"
I20260808 18:26:35.733915 10736 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 18:26:35.736742 10750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:35.737993 10736 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:26:35.738109 10736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-2/wal
uuid: "dcc484db0c6545c4b9b7cc2d2689776c"
format_stamp: "Formatted at 2026-08-08 18:26:35 on dist-test-slave-5k54"
server_key: "e3156f8da6e6c16d1bf739e195be5887"
server_key_iv: "ef3d6a5737c95bc6ec9af61626450e28"
server_key_version: "encryptionkey@0"
I20260808 18:26:35.738212 10736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:26:35.768343 10736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:26:35.769096 10736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:26:35.769296 10736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:26:35.769969 10736 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:26:35.770934 10736 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:26:35.770991 10736 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:35.771099 10736 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:26:35.771126 10736 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:35.780143 10736 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.131:42645
I20260808 18:26:35.780180 10875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.131:42645 every 8 connection(s)
I20260808 18:26:35.781037 10736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
I20260808 18:26:35.782740 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 10736
I20260808 18:26:35.782869 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance
I20260808 18:26:35.783185 23842 external_mini_cluster.cc:1428] Setting key c93f45a78ccceb4731dd13cbbf9472ad
I20260808 18:26:35.790211 10439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:35.782823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:44797 (local address 127.23.72.190:45781)
0808 18:26:35.782997 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:35.783003 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:35.784188 (+ 1185us) server_negotiation.cc:407] Connection header received
0808 18:26:35.785782 (+ 1594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:35.785785 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:35.785828 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:35.785895 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:35.786688 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:35.787194 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:35.788360 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:35.788582 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:35.788688 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:35.789774 (+ 1086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:35.789777 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:35.789779 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:26:35.789790 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:35.789809 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:26:35.789817 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:35.789877 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:35.790035 (+ 158us) server_negotiation.cc:299] Negotiation successful
0808 18:26:35.790088 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 18:26:35.791021 10878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:35.783255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:45781 (local address 127.23.72.131:44797)
0808 18:26:35.784037 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:35.784092 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:35.785586 (+ 1494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:35.786020 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:35.786026 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:35.786104 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:26:35.786573 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:35.786586 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:35.787305 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:35.787307 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:35.788227 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:35.788237 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:35.788738 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:35.788741 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:35.788919 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:35.789487 (+ 568us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:26:35.789491 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:26:35.789547 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:26:35.789632 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:26:35.789900 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:26:35.789907 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:26:35.789911 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 18:26:35.790004 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 18:26:35.790221 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":166,"threads_started":1}
I20260808 18:26:35.792100 10876 heartbeater.cc:347] Connected to a master server at 127.23.72.190:45781
I20260808 18:26:35.792325 10876 heartbeater.cc:464] Registering TS with master...
I20260808 18:26:35.792820 10876 heartbeater.cc:511] Master 127.23.72.190:45781 requested a full tablet report, sending...
I20260808 18:26:35.793802 10395 ts_manager.cc:194] Registered new tserver with Master: dcc484db0c6545c4b9b7cc2d2689776c (127.23.72.131:42645)
I20260808 18:26:35.794342 10395 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.72.131:44797
I20260808 18:26:35.796943 23842 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:26:35.802325 10439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:35.798783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48448 (local address 127.23.72.190:45781)
0808 18:26:35.798895 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:35.798899 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:35.799090 (+ 191us) server_negotiation.cc:407] Connection header received
0808 18:26:35.799207 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:35.799209 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:35.799243 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:35.799306 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:35.799618 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:35.800072 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:35.800883 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:35.801077 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:35.801148 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:35.801990 (+ 842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:35.801992 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:35.801994 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:26:35.802005 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:35.802021 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:26:35.802028 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:35.802083 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:35.802191 (+ 108us) server_negotiation.cc:299] Negotiation successful
0808 18:26:35.802234 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 18:26:35.804983 10395 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48448:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:26:35.807964 10395 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:26:35.822156 10890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:35.816378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.130:44991 (local address 127.0.0.1:51710)
0808 18:26:35.817483 (+ 1105us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:35.817496 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:35.817589 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:35.818043 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:35.818046 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:35.818089 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:26:35.818257 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:35.818265 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:35.819775 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:35.819785 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:35.820844 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:35.820853 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:35.821618 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:35.821621 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:35.821722 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:35.821758 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 18:26:35.821878 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 18:26:35.822000 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1031,"mutex_wait_us":103,"spinlock_wait_cycles":5632,"thread_start_us":104,"threads_started":1}
I20260808 18:26:35.822193 10894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:35.816799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51710 (local address 127.23.72.130:44991)
0808 18:26:35.817328 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:35.817342 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:35.817586 (+ 244us) server_negotiation.cc:407] Connection header received
0808 18:26:35.817685 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:35.817688 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:35.817881 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:35.818430 (+ 549us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:26:35.818565 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:35.819640 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:35.820962 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:35.821492 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:35.821607 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:35.821795 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:35.821952 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 18:26:35.822050 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":165,"threads_started":1}
I20260808 18:26:35.822398 10889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:35.816114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:44435 (local address 127.0.0.1:34562)
0808 18:26:35.817111 (+ 997us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:35.817160 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:35.817284 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:35.818040 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:35.818057 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:35.818092 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:26:35.818325 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:35.818335 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:35.819887 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:35.819891 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:35.820859 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:35.820871 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:35.822017 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:35.822021 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:35.822110 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:35.822135 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 18:26:35.822182 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 18:26:35.822251 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":821,"mutex_wait_us":120,"spinlock_wait_cycles":3328,"thread_start_us":79,"threads_started":1}
I20260808 18:26:35.823155 10892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:35.816014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.131:42645 (local address 127.0.0.1:37430)
0808 18:26:35.817712 (+ 1698us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:35.817724 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:35.817798 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:35.818229 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:35.818232 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:35.818249 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:26:35.818377 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:35.818386 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:35.819367 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:35.819373 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:35.821496 (+ 2123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:35.821506 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:35.822792 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:35.822796 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:35.822877 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:35.822898 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 18:26:35.822968 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 18:26:35.823036 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":900,"mutex_wait_us":69,"thread_start_us":66,"threads_started":1}
I20260808 18:26:35.823390 10893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:35.816786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34562 (local address 127.23.72.129:44435)
0808 18:26:35.817534 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:35.817548 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:35.817575 (+ 27us) server_negotiation.cc:407] Connection header received
0808 18:26:35.817656 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:35.817663 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:35.817876 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:35.818051 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:26:35.818462 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:35.819768 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:35.821012 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:35.821858 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:35.822807 (+ 949us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:35.823000 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:35.823049 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 18:26:35.823140 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":157,"threads_started":1}
I20260808 18:26:35.823655 10891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:35.816139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37430 (local address 127.23.72.131:42645)
0808 18:26:35.816822 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:35.816835 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:35.817808 (+ 973us) server_negotiation.cc:407] Connection header received
0808 18:26:35.817883 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:35.817888 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:35.818051 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:35.818189 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:26:35.818509 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:35.819245 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:35.821641 (+ 2396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:35.822394 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:35.822515 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:35.822728 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:35.823335 (+ 607us) server_negotiation.cc:299] Negotiation successful
0808 18:26:35.823474 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":163,"threads_started":1}
I20260808 18:26:35.825459 10664 tablet_service.cc:1467] Processing CreateTablet for tablet e96530d859284b95b0180fbc5996aded (DEFAULT_TABLE table=test-table [id=250c9f61e46c41e4950d458406a35958]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:26:35.826457 10664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e96530d859284b95b0180fbc5996aded. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:26:35.826188 10525 tablet_service.cc:1467] Processing CreateTablet for tablet e96530d859284b95b0180fbc5996aded (DEFAULT_TABLE table=test-table [id=250c9f61e46c41e4950d458406a35958]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:26:35.826188 10807 tablet_service.cc:1467] Processing CreateTablet for tablet e96530d859284b95b0180fbc5996aded (DEFAULT_TABLE table=test-table [id=250c9f61e46c41e4950d458406a35958]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:26:35.827225 10525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e96530d859284b95b0180fbc5996aded. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:26:35.827225 10807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e96530d859284b95b0180fbc5996aded. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:26:35.832664 10895 tablet_bootstrap.cc:492] T e96530d859284b95b0180fbc5996aded P bbc22d236cfe42cf81727c9f6c826b63: Bootstrap starting.
I20260808 18:26:35.834177 10896 tablet_bootstrap.cc:492] T e96530d859284b95b0180fbc5996aded P dcc484db0c6545c4b9b7cc2d2689776c: Bootstrap starting.
I20260808 18:26:35.835600 10895 tablet_bootstrap.cc:654] T e96530d859284b95b0180fbc5996aded P bbc22d236cfe42cf81727c9f6c826b63: Neither blocks nor log segments found. Creating new log.
I20260808 18:26:35.835765 10897 tablet_bootstrap.cc:492] T e96530d859284b95b0180fbc5996aded P 124426d2d97049e786f93096f8ca3957: Bootstrap starting.
I20260808 18:26:35.836347 10896 tablet_bootstrap.cc:654] T e96530d859284b95b0180fbc5996aded P dcc484db0c6545c4b9b7cc2d2689776c: Neither blocks nor log segments found. Creating new log.
I20260808 18:26:35.836484 10895 log.cc:824] T e96530d859284b95b0180fbc5996aded P bbc22d236cfe42cf81727c9f6c826b63: Log is configured to *not* fsync() on all Append() calls
I20260808 18:26:35.837219 10896 log.cc:824] T e96530d859284b95b0180fbc5996aded P dcc484db0c6545c4b9b7cc2d2689776c: Log is configured to *not* fsync() on all Append() calls
I20260808 18:26:35.837769 10897 tablet_bootstrap.cc:654] T e96530d859284b95b0180fbc5996aded P 124426d2d97049e786f93096f8ca3957: Neither blocks nor log segments found. Creating new log.
I20260808 18:26:35.838599 10897 log.cc:824] T e96530d859284b95b0180fbc5996aded P 124426d2d97049e786f93096f8ca3957: Log is configured to *not* fsync() on all Append() calls
I20260808 18:26:35.839030 10895 tablet_bootstrap.cc:492] T e96530d859284b95b0180fbc5996aded P bbc22d236cfe42cf81727c9f6c826b63: No bootstrap required, opened a new log
I20260808 18:26:35.839201 10896 tablet_bootstrap.cc:492] T e96530d859284b95b0180fbc5996aded P dcc484db0c6545c4b9b7cc2d2689776c: No bootstrap required, opened a new log
I20260808 18:26:35.839231 10895 ts_tablet_manager.cc:1397] T e96530d859284b95b0180fbc5996aded P bbc22d236cfe42cf81727c9f6c826b63: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260808 18:26:35.839392 10896 ts_tablet_manager.cc:1397] T e96530d859284b95b0180fbc5996aded P dcc484db0c6545c4b9b7cc2d2689776c: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 18:26:35.840660 10897 tablet_bootstrap.cc:492] T e96530d859284b95b0180fbc5996aded P 124426d2d97049e786f93096f8ca3957: No bootstrap required, opened a new log
I20260808 18:26:35.840904 10897 ts_tablet_manager.cc:1397] T e96530d859284b95b0180fbc5996aded P 124426d2d97049e786f93096f8ca3957: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260808 18:26:35.842901 10895 raft_consensus.cc:348] T e96530d859284b95b0180fbc5996aded P bbc22d236cfe42cf81727c9f6c826b63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcc484db0c6545c4b9b7cc2d2689776c" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 42645 } } peers { permanent_uuid: "124426d2d97049e786f93096f8ca3957" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44435 } } peers { permanent_uuid: "bbc22d236cfe42cf81727c9f6c826b63" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 44991 } }
I20260808 18:26:35.843132 10895 raft_consensus.cc:374] T e96530d859284b95b0180fbc5996aded P bbc22d236cfe42cf81727c9f6c826b63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:26:35.843189 10895 raft_consensus.cc:729] T e96530d859284b95b0180fbc5996aded P bbc22d236cfe42cf81727c9f6c826b63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbc22d236cfe42cf81727c9f6c826b63, State: Initialized, Role: FOLLOWER
I20260808 18:26:35.843649 10895 consensus_queue.cc:260] T e96530d859284b95b0180fbc5996aded P bbc22d236cfe42cf81727c9f6c826b63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcc484db0c6545c4b9b7cc2d2689776c" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 42645 } } peers { permanent_uuid: "124426d2d97049e786f93096f8ca3957" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44435 } } peers { permanent_uuid: "bbc22d236cfe42cf81727c9f6c826b63" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 44991 } }
I20260808 18:26:35.843778 10896 raft_consensus.cc:348] T e96530d859284b95b0180fbc5996aded P dcc484db0c6545c4b9b7cc2d2689776c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcc484db0c6545c4b9b7cc2d2689776c" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 42645 } } peers { permanent_uuid: "124426d2d97049e786f93096f8ca3957" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44435 } } peers { permanent_uuid: "bbc22d236cfe42cf81727c9f6c826b63" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 44991 } }
I20260808 18:26:35.844018 10896 raft_consensus.cc:374] T e96530d859284b95b0180fbc5996aded P dcc484db0c6545c4b9b7cc2d2689776c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:26:35.844077 10896 raft_consensus.cc:729] T e96530d859284b95b0180fbc5996aded P dcc484db0c6545c4b9b7cc2d2689776c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcc484db0c6545c4b9b7cc2d2689776c, State: Initialized, Role: FOLLOWER
I20260808 18:26:35.844532 10896 consensus_queue.cc:260] T e96530d859284b95b0180fbc5996aded P dcc484db0c6545c4b9b7cc2d2689776c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcc484db0c6545c4b9b7cc2d2689776c" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 42645 } } peers { permanent_uuid: "124426d2d97049e786f93096f8ca3957" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44435 } } peers { permanent_uuid: "bbc22d236cfe42cf81727c9f6c826b63" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 44991 } }
I20260808 18:26:35.844980 10895 ts_tablet_manager.cc:1428] T e96530d859284b95b0180fbc5996aded P bbc22d236cfe42cf81727c9f6c826b63: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 18:26:35.845080 10897 raft_consensus.cc:348] T e96530d859284b95b0180fbc5996aded P 124426d2d97049e786f93096f8ca3957 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcc484db0c6545c4b9b7cc2d2689776c" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 42645 } } peers { permanent_uuid: "124426d2d97049e786f93096f8ca3957" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44435 } } peers { permanent_uuid: "bbc22d236cfe42cf81727c9f6c826b63" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 44991 } }
I20260808 18:26:35.845245 10876 heartbeater.cc:503] Master 127.23.72.190:45781 was elected leader, sending a full tablet report...
I20260808 18:26:35.845245 10733 heartbeater.cc:503] Master 127.23.72.190:45781 was elected leader, sending a full tablet report...
I20260808 18:26:35.845319 10897 raft_consensus.cc:374] T e96530d859284b95b0180fbc5996aded P 124426d2d97049e786f93096f8ca3957 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:26:35.845376 10897 raft_consensus.cc:729] T e96530d859284b95b0180fbc5996aded P 124426d2d97049e786f93096f8ca3957 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 124426d2d97049e786f93096f8ca3957, State: Initialized, Role: FOLLOWER
I20260808 18:26:35.845502 10896 ts_tablet_manager.cc:1428] T e96530d859284b95b0180fbc5996aded P dcc484db0c6545c4b9b7cc2d2689776c: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 18:26:35.845799 10897 consensus_queue.cc:260] T e96530d859284b95b0180fbc5996aded P 124426d2d97049e786f93096f8ca3957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcc484db0c6545c4b9b7cc2d2689776c" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 42645 } } peers { permanent_uuid: "124426d2d97049e786f93096f8ca3957" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44435 } } peers { permanent_uuid: "bbc22d236cfe42cf81727c9f6c826b63" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 44991 } }
I20260808 18:26:35.846374 10595 heartbeater.cc:503] Master 127.23.72.190:45781 was elected leader, sending a full tablet report...
I20260808 18:26:35.846649 10897 ts_tablet_manager.cc:1428] T e96530d859284b95b0180fbc5996aded P 124426d2d97049e786f93096f8ca3957: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260808 18:26:35.890476 10596 tablet.cc:2369] T e96530d859284b95b0180fbc5996aded P 124426d2d97049e786f93096f8ca3957: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:26:35.957959 10902 raft_consensus.cc:482] T e96530d859284b95b0180fbc5996aded P dcc484db0c6545c4b9b7cc2d2689776c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:26:35.958200 10902 raft_consensus.cc:504] T e96530d859284b95b0180fbc5996aded P dcc484db0c6545c4b9b7cc2d2689776c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcc484db0c6545c4b9b7cc2d2689776c" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 42645 } } peers { permanent_uuid: "124426d2d97049e786f93096f8ca3957" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44435 } } peers { permanent_uuid: "bbc22d236cfe42cf81727c9f6c826b63" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 44991 } }
I20260808 18:26:35.959522 10902 leader_election.cc:288] T e96530d859284b95b0180fbc5996aded P dcc484db0c6545c4b9b7cc2d2689776c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 124426d2d97049e786f93096f8ca3957 (127.23.72.129:44435), bbc22d236cfe42cf81727c9f6c826b63 (127.23.72.130:44991)
I20260808 18:26:35.964704 10878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:35.959752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:44435 (local address 127.23.72.131:39895)
0808 18:26:35.959994 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:35.960010 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:35.960125 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:35.960491 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:35.960494 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:35.960534 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:26:35.960694 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:35.960701 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:35.961746 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:35.961749 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:35.963249 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:35.963261 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:35.964192 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:35.964199 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:35.964305 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:35.964380 (+ 75us) client_negotiation.cc:769] Sending connection context
0808 18:26:35.964459 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 18:26:35.964539 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":196,"spinlock_wait_cycles":6528}
I20260808 18:26:35.964946 10904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:35.959798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.130:44991 (local address 127.23.72.131:58753)
0808 18:26:35.960166 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:35.960180 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:35.960269 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:35.960492 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:35.960494 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:35.960532 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:26:35.960699 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:35.960704 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:35.961746 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:35.961749 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:35.963249 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:35.963261 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:35.964312 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:35.964316 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:35.964464 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:35.964489 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 18:26:35.964571 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 18:26:35.964776 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"mutex_wait_us":166,"spinlock_wait_cycles":30976,"thread_start_us":115,"threads_started":1}
I20260808 18:26:35.965142 10894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:35.959892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:58753 (local address 127.23.72.130:44991)
0808 18:26:35.960030 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:35.960037 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:35.960262 (+ 225us) server_negotiation.cc:407] Connection header received
0808 18:26:35.960342 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:35.960345 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:35.960400 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:35.960476 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:26:35.960876 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:35.961627 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:35.963452 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:35.964097 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:35.964202 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:35.964242 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:35.964610 (+ 368us) server_negotiation.cc:299] Negotiation successful
0808 18:26:35.964720 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 18:26:35.965399 10893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:35.959880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:39895 (local address 127.23.72.129:44435)
0808 18:26:35.959994 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:35.960000 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:35.960109 (+ 109us) server_negotiation.cc:407] Connection header received
0808 18:26:35.960269 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:35.960299 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:35.960363 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:35.960450 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:26:35.960827 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:35.961636 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:35.963456 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:35.964049 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:35.964192 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:35.964242 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:35.965166 (+ 924us) server_negotiation.cc:299] Negotiation successful
0808 18:26:35.965250 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 18:26:35.965762 10684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e96530d859284b95b0180fbc5996aded" candidate_uuid: "dcc484db0c6545c4b9b7cc2d2689776c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbc22d236cfe42cf81727c9f6c826b63" is_pre_election: true
I20260808 18:26:35.965955 10546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e96530d859284b95b0180fbc5996aded" candidate_uuid: "dcc484db0c6545c4b9b7cc2d2689776c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "124426d2d97049e786f93096f8ca3957" is_pre_election: true
I20260808 18:26:35.966090 10684 raft_consensus.cc:2454] T e96530d859284b95b0180fbc5996aded P bbc22d236cfe42cf81727c9f6c826b63 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dcc484db0c6545c4b9b7cc2d2689776c in term 0.
I20260808 18:26:35.966253 10546 raft_consensus.cc:2454] T e96530d859284b95b0180fbc5996aded P 124426d2d97049e786f93096f8ca3957 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dcc484db0c6545c4b9b7cc2d2689776c in term 0.
I20260808 18:26:35.966674 10751 leader_election.cc:302] T e96530d859284b95b0180fbc5996aded P dcc484db0c6545c4b9b7cc2d2689776c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bbc22d236cfe42cf81727c9f6c826b63, dcc484db0c6545c4b9b7cc2d2689776c; no voters:
I20260808 18:26:35.966987 10902 raft_consensus.cc:2781] T e96530d859284b95b0180fbc5996aded P dcc484db0c6545c4b9b7cc2d2689776c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:26:35.967077 10902 raft_consensus.cc:482] T e96530d859284b95b0180fbc5996aded P dcc484db0c6545c4b9b7cc2d2689776c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:26:35.967121 10902 raft_consensus.cc:3037] T e96530d859284b95b0180fbc5996aded P dcc484db0c6545c4b9b7cc2d2689776c [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:26:35.968209 10902 raft_consensus.cc:504] T e96530d859284b95b0180fbc5996aded P dcc484db0c6545c4b9b7cc2d2689776c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcc484db0c6545c4b9b7cc2d2689776c" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 42645 } } peers { permanent_uuid: "124426d2d97049e786f93096f8ca3957" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44435 } } peers { permanent_uuid: "bbc22d236cfe42cf81727c9f6c826b63" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 44991 } }
I20260808 18:26:35.968559 10902 leader_election.cc:288] T e96530d859284b95b0180fbc5996aded P dcc484db0c6545c4b9b7cc2d2689776c [CANDIDATE]: Term 1 election: Requested vote from peers 124426d2d97049e786f93096f8ca3957 (127.23.72.129:44435), bbc22d236cfe42cf81727c9f6c826b63 (127.23.72.130:44991)
I20260808 18:26:35.968899 10546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e96530d859284b95b0180fbc5996aded" candidate_uuid: "dcc484db0c6545c4b9b7cc2d2689776c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "124426d2d97049e786f93096f8ca3957"
I20260808 18:26:35.968945 10684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e96530d859284b95b0180fbc5996aded" candidate_uuid: "dcc484db0c6545c4b9b7cc2d2689776c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbc22d236cfe42cf81727c9f6c826b63"
I20260808 18:26:35.969048 10546 raft_consensus.cc:3037] T e96530d859284b95b0180fbc5996aded P 124426d2d97049e786f93096f8ca3957 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:26:35.969064 10684 raft_consensus.cc:3037] T e96530d859284b95b0180fbc5996aded P bbc22d236cfe42cf81727c9f6c826b63 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:26:35.970072 10684 raft_consensus.cc:2454] T e96530d859284b95b0180fbc5996aded P bbc22d236cfe42cf81727c9f6c826b63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dcc484db0c6545c4b9b7cc2d2689776c in term 1.
I20260808 18:26:35.970180 10546 raft_consensus.cc:2454] T e96530d859284b95b0180fbc5996aded P 124426d2d97049e786f93096f8ca3957 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dcc484db0c6545c4b9b7cc2d2689776c in term 1.
I20260808 18:26:35.970418 10751 leader_election.cc:302] T e96530d859284b95b0180fbc5996aded P dcc484db0c6545c4b9b7cc2d2689776c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bbc22d236cfe42cf81727c9f6c826b63, dcc484db0c6545c4b9b7cc2d2689776c; no voters:
I20260808 18:26:35.970619 10902 raft_consensus.cc:2781] T e96530d859284b95b0180fbc5996aded P dcc484db0c6545c4b9b7cc2d2689776c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:26:35.970832 10902 raft_consensus.cc:686] T e96530d859284b95b0180fbc5996aded P dcc484db0c6545c4b9b7cc2d2689776c [term 1 LEADER]: Becoming Leader. State: Replica: dcc484db0c6545c4b9b7cc2d2689776c, State: Running, Role: LEADER
I20260808 18:26:35.971128 10902 consensus_queue.cc:237] T e96530d859284b95b0180fbc5996aded P dcc484db0c6545c4b9b7cc2d2689776c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcc484db0c6545c4b9b7cc2d2689776c" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 42645 } } peers { permanent_uuid: "124426d2d97049e786f93096f8ca3957" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44435 } } peers { permanent_uuid: "bbc22d236cfe42cf81727c9f6c826b63" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 44991 } }
I20260808 18:26:35.974438 10395 catalog_manager.cc:5495] T e96530d859284b95b0180fbc5996aded P dcc484db0c6545c4b9b7cc2d2689776c reported cstate change: term changed from 0 to 1, leader changed from <none> to dcc484db0c6545c4b9b7cc2d2689776c (127.23.72.131). New cstate: current_term: 1 leader_uuid: "dcc484db0c6545c4b9b7cc2d2689776c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcc484db0c6545c4b9b7cc2d2689776c" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 42645 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "124426d2d97049e786f93096f8ca3957" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bbc22d236cfe42cf81727c9f6c826b63" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 44991 } health_report { overall_health: UNKNOWN } } }
I20260808 18:26:36.009864 10891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:36.006309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37436 (local address 127.23.72.131:42645)
0808 18:26:36.006479 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:36.006484 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:36.006499 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:26:36.006619 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:36.006621 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:36.006676 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:36.006792 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:26:36.007059 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:36.007575 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:36.008408 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:36.008623 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:36.008756 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:36.009066 (+ 310us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:26:36.009476 (+ 410us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:26:36.009610 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:36.009671 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 18:26:36.009720 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260808 18:26:36.015765 10684 raft_consensus.cc:1261] T e96530d859284b95b0180fbc5996aded P bbc22d236cfe42cf81727c9f6c826b63 [term 1 FOLLOWER]: Refusing update from remote peer dcc484db0c6545c4b9b7cc2d2689776c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:26:36.015765 10546 raft_consensus.cc:1261] T e96530d859284b95b0180fbc5996aded P 124426d2d97049e786f93096f8ca3957 [term 1 FOLLOWER]: Refusing update from remote peer dcc484db0c6545c4b9b7cc2d2689776c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:26:36.016523 10905 consensus_queue.cc:1035] T e96530d859284b95b0180fbc5996aded P dcc484db0c6545c4b9b7cc2d2689776c [LEADER]: Connected to new peer: Peer: permanent_uuid: "124426d2d97049e786f93096f8ca3957" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 44435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:26:36.016673 10902 consensus_queue.cc:1035] T e96530d859284b95b0180fbc5996aded P dcc484db0c6545c4b9b7cc2d2689776c [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbc22d236cfe42cf81727c9f6c826b63" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 44991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:26:36.023610 10909 mvcc.cc:205] Tried to move back new op lower bound from 7316330889272954880 to 7316330889100365824. Current Snapshot: MvccSnapshot[applied={T|T < 7316330889272954880}]
I20260808 18:26:36.023813 10911 mvcc.cc:205] Tried to move back new op lower bound from 7316330889272954880 to 7316330889100365824. Current Snapshot: MvccSnapshot[applied={T|T < 7316330889272954880}]
I20260808 18:26:36.025405 10910 mvcc.cc:205] Tried to move back new op lower bound from 7316330889272954880 to 7316330889100365824. Current Snapshot: MvccSnapshot[applied={T|T < 7316330889272954880}]
I20260808 18:26:36.032238 10395 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48448:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:26:36.032407 10395 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48448:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:26:36.034752 10395 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d2a3c733c5ce42a49024e2ebe0f6b393: Sending DeleteTablet for 3 replicas of tablet e96530d859284b95b0180fbc5996aded
I20260808 18:26:36.035509 10807 tablet_service.cc:1514] Processing DeleteTablet for tablet e96530d859284b95b0180fbc5996aded with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:26:36 UTC) from {username='slave'} at 127.0.0.1:37430
I20260808 18:26:36.035518 10525 tablet_service.cc:1514] Processing DeleteTablet for tablet e96530d859284b95b0180fbc5996aded with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:26:36 UTC) from {username='slave'} at 127.0.0.1:34562
I20260808 18:26:36.035609 10664 tablet_service.cc:1514] Processing DeleteTablet for tablet e96530d859284b95b0180fbc5996aded with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:26:36 UTC) from {username='slave'} at 127.0.0.1:51710
I20260808 18:26:36.036034 10921 tablet_replica.cc:330] T e96530d859284b95b0180fbc5996aded P dcc484db0c6545c4b9b7cc2d2689776c: stopping tablet replica
I20260808 18:26:36.036355 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 10461
I20260808 18:26:36.036342 10921 raft_consensus.cc:2229] T e96530d859284b95b0180fbc5996aded P dcc484db0c6545c4b9b7cc2d2689776c [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:26:36.036664 10921 raft_consensus.cc:2258] T e96530d859284b95b0180fbc5996aded P dcc484db0c6545c4b9b7cc2d2689776c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:26:36.038304 10921 ts_tablet_manager.cc:1905] T e96530d859284b95b0180fbc5996aded P dcc484db0c6545c4b9b7cc2d2689776c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:26:36.040686 10921 ts_tablet_manager.cc:1918] T e96530d859284b95b0180fbc5996aded P dcc484db0c6545c4b9b7cc2d2689776c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:26:36.040795 10921 log.cc:1196] T e96530d859284b95b0180fbc5996aded P dcc484db0c6545c4b9b7cc2d2689776c: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-2/wal/wals/e96530d859284b95b0180fbc5996aded
I20260808 18:26:36.041165 10921 ts_tablet_manager.cc:1939] T e96530d859284b95b0180fbc5996aded P dcc484db0c6545c4b9b7cc2d2689776c: Deleting consensus metadata
I20260808 18:26:36.042265 10923 tablet_replica.cc:330] T e96530d859284b95b0180fbc5996aded P bbc22d236cfe42cf81727c9f6c826b63: stopping tablet replica
I20260808 18:26:36.042245 10380 catalog_manager.cc:4841] TS dcc484db0c6545c4b9b7cc2d2689776c (127.23.72.131:42645): tablet e96530d859284b95b0180fbc5996aded (table test-table [id=250c9f61e46c41e4950d458406a35958]) successfully deleted
I20260808 18:26:36.042567 10923 raft_consensus.cc:2229] T e96530d859284b95b0180fbc5996aded P bbc22d236cfe42cf81727c9f6c826b63 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:26:36.042861 10923 raft_consensus.cc:2258] T e96530d859284b95b0180fbc5996aded P bbc22d236cfe42cf81727c9f6c826b63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:26:36.044225 10923 ts_tablet_manager.cc:1905] T e96530d859284b95b0180fbc5996aded P bbc22d236cfe42cf81727c9f6c826b63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:26:36.046233 10923 ts_tablet_manager.cc:1918] T e96530d859284b95b0180fbc5996aded P bbc22d236cfe42cf81727c9f6c826b63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:26:36.046343 10923 log.cc:1196] T e96530d859284b95b0180fbc5996aded P bbc22d236cfe42cf81727c9f6c826b63: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213345431840-23842-0/minicluster-data/ts-1/wal/wals/e96530d859284b95b0180fbc5996aded
I20260808 18:26:36.046623 10923 ts_tablet_manager.cc:1939] T e96530d859284b95b0180fbc5996aded P bbc22d236cfe42cf81727c9f6c826b63: Deleting consensus metadata
W20260808 18:26:36.047415 10378 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.72.129:44435 (error 108)
I20260808 18:26:36.047646 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 10598
I20260808 18:26:36.048146 10377 catalog_manager.cc:4841] TS bbc22d236cfe42cf81727c9f6c826b63 (127.23.72.130:44991): tablet e96530d859284b95b0180fbc5996aded (table test-table [id=250c9f61e46c41e4950d458406a35958]) successfully deleted
I20260808 18:26:36.048979 10892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:36.048616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:44435 (local address 127.0.0.1:34568)
0808 18:26:36.048803 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:36.048877 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.129:44435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
W20260808 18:26:36.049202 10378 catalog_manager.cc:4584] TS 124426d2d97049e786f93096f8ca3957 (127.23.72.129:44435): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e96530d859284b95b0180fbc5996aded: Network error: Client connection negotiation failed: client connection to 127.23.72.129:44435: connect: Connection refused (error 111)
I20260808 18:26:36.054481 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 10736
I20260808 18:26:36.061329 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 10362
2026-08-08T18:26:36Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1138 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:26:36 dist-test-slave-5k54 krb5kdc[10928](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:26:36 dist-test-slave-5k54 krb5kdc[10928](info): set up 2 sockets
Aug 08 18:26:36 dist-test-slave-5k54 krb5kdc[10928](info): commencing operation
krb5kdc: starting...
W20260808 18:26:37.139660 23842 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Aug 08 18:26:37 dist-test-slave-5k54 krb5kdc[10928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213597, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:26:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:26:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.72.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
I20260808 18:26:37.270800 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:40091
--webserver_interface=127.23.72.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:43907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:40091
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:26:37.381191 10941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:26:37.381497 10941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:26:37.381572 10941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:26:37.385565 10941 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:26:37.385651 10941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:26:37.385682 10941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:26:37.385711 10941 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:26:37.385748 10941 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:26:37.390904 10941 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:43907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:40091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:40091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.72.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10941
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:26:37.391978 10941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:26:37.392994 10941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:26:37.401718 10947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:37.401700 10941 server_base.cc:1034] running on GCE node
W20260808 18:26:37.401690 10951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:37.401722 10949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:37.402455 10941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:26:37.403599 10941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:26:37.404790 10941 hybrid_clock.cc:648] HybridClock initialized: now 1786213597404770 us; error 31 us; skew 500 ppm
Aug 08 18:26:37 dist-test-slave-5k54 krb5kdc[10928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213597, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:26:37.407655 10941 init.cc:377] Logged in from keytab as kudu/127.23.72.190@KRBTEST.COM (short username kudu)
I20260808 18:26:37.408447 10941 webserver.cc:460] Webserver started at http://127.23.72.190:35505/ using document root <none> and password file <none>
I20260808 18:26:37.408952 10941 fs_manager.cc:359] Metadata directory not provided
I20260808 18:26:37.409008 10941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:26:37.409162 10941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:26:37.411038 10941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "2f667c524c63416cafbb443054a412b5"
format_stamp: "Formatted at 2026-08-08 18:26:37 on dist-test-slave-5k54"
server_key: "449c5fb215376af4f00918551690b446"
server_key_iv: "1592cfa6abed517991b057c34a08cb53"
server_key_version: "encryptionkey@0"
I20260808 18:26:37.411631 10941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "2f667c524c63416cafbb443054a412b5"
format_stamp: "Formatted at 2026-08-08 18:26:37 on dist-test-slave-5k54"
server_key: "449c5fb215376af4f00918551690b446"
server_key_iv: "1592cfa6abed517991b057c34a08cb53"
server_key_version: "encryptionkey@0"
I20260808 18:26:37.415154 10941 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:26:37.417591 10958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:37.418777 10941 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:26:37.418879 10941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "2f667c524c63416cafbb443054a412b5"
format_stamp: "Formatted at 2026-08-08 18:26:37 on dist-test-slave-5k54"
server_key: "449c5fb215376af4f00918551690b446"
server_key_iv: "1592cfa6abed517991b057c34a08cb53"
server_key_version: "encryptionkey@0"
I20260808 18:26:37.418960 10941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:26:37.429608 10941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:26:37.432543 10941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:26:37.432705 10941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:26:37.439714 10941 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.190:40091
I20260808 18:26:37.439790 11011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.190:40091 every 8 connection(s)
I20260808 18:26:37.440697 10941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:26:37.443399 11012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:26:37.447502 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 10941
I20260808 18:26:37.447659 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:26:37.447981 23842 external_mini_cluster.cc:1428] Setting key 6eb675983f1d40deda23327f3cba9e6c
I20260808 18:26:37.449221 11012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f667c524c63416cafbb443054a412b5: Bootstrap starting.
I20260808 18:26:37.451583 11012 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f667c524c63416cafbb443054a412b5: Neither blocks nor log segments found. Creating new log.
I20260808 18:26:37.452476 11012 log.cc:824] T 00000000000000000000000000000000 P 2f667c524c63416cafbb443054a412b5: Log is configured to *not* fsync() on all Append() calls
I20260808 18:26:37.454595 11012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f667c524c63416cafbb443054a412b5: No bootstrap required, opened a new log
Aug 08 18:26:37 dist-test-slave-5k54 krb5kdc[10928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213597, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:26:37.458841 11012 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f667c524c63416cafbb443054a412b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f667c524c63416cafbb443054a412b5" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40091 } }
I20260808 18:26:37.459080 11012 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f667c524c63416cafbb443054a412b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:26:37.459156 11012 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f667c524c63416cafbb443054a412b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f667c524c63416cafbb443054a412b5, State: Initialized, Role: FOLLOWER
I20260808 18:26:37.459640 11012 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f667c524c63416cafbb443054a412b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f667c524c63416cafbb443054a412b5" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40091 } }
I20260808 18:26:37.459800 11012 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f667c524c63416cafbb443054a412b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:26:37.459868 11012 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f667c524c63416cafbb443054a412b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:26:37.459964 11012 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f667c524c63416cafbb443054a412b5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:26:37.460836 11012 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f667c524c63416cafbb443054a412b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f667c524c63416cafbb443054a412b5" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40091 } }
I20260808 18:26:37.461179 11012 leader_election.cc:302] T 00000000000000000000000000000000 P 2f667c524c63416cafbb443054a412b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f667c524c63416cafbb443054a412b5; no voters:
I20260808 18:26:37.461519 11012 leader_election.cc:288] T 00000000000000000000000000000000 P 2f667c524c63416cafbb443054a412b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:26:37.461640 11017 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f667c524c63416cafbb443054a412b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:26:37.461879 11017 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f667c524c63416cafbb443054a412b5 [term 1 LEADER]: Becoming Leader. State: Replica: 2f667c524c63416cafbb443054a412b5, State: Running, Role: LEADER
I20260808 18:26:37.462234 11017 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f667c524c63416cafbb443054a412b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f667c524c63416cafbb443054a412b5" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40091 } }
I20260808 18:26:37.462735 11015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:37.449434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51358 (local address 127.23.72.190:40091)
0808 18:26:37.449835 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:37.449852 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:37.449887 (+ 35us) server_negotiation.cc:407] Connection header received
0808 18:26:37.450507 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:37.450524 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:37.450837 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:37.451099 (+ 262us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:37.452156 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:37.453198 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:37.454345 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:37.454858 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:37.454975 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:37.457724 (+ 2749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:37.457729 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:37.457742 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:26:37.457768 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:37.459639 (+ 1871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:37.460146 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:37.460151 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:37.460155 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:37.460261 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:37.460544 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:37.460548 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:37.460550 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:37.460983 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:37.461174 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:37.461472 (+ 298us) server_negotiation.cc:299] Negotiation successful
0808 18:26:37.461819 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":130,"threads_started":1}
I20260808 18:26:37.462813 11012 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f667c524c63416cafbb443054a412b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:26:37.464161 11018 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f667c524c63416cafbb443054a412b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f667c524c63416cafbb443054a412b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f667c524c63416cafbb443054a412b5" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40091 } } }
I20260808 18:26:37.464298 11018 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f667c524c63416cafbb443054a412b5 [sys.catalog]: This master's current role is: LEADER
I20260808 18:26:37.464704 11027 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:26:37.466264 11019 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f667c524c63416cafbb443054a412b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f667c524c63416cafbb443054a412b5. Latest consensus state: current_term: 1 leader_uuid: "2f667c524c63416cafbb443054a412b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f667c524c63416cafbb443054a412b5" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40091 } } }
I20260808 18:26:37.466368 11019 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f667c524c63416cafbb443054a412b5 [sys.catalog]: This master's current role is: LEADER
I20260808 18:26:37.467734 11027 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:26:37.473733 11027 catalog_manager.cc:1347] Generated new cluster ID: c9f8bda08b09412b8ac58fc04dd21bec
I20260808 18:26:37.473845 11027 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:26:37.488327 11027 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:26:37.489463 11027 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:26:37.496711 11027 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f667c524c63416cafbb443054a412b5: Generated new TSK 0
I20260808 18:26:37.497491 11027 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.72.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab.
Entry for principal kudu/127.23.72.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab.
I20260808 18:26:37.534524 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.129:0
--local_ip_for_outbound_sockets=127.23.72.129
--webserver_interface=127.23.72.129
--webserver_port=0
--tserver_master_addrs=127.23.72.190:40091
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab
--principal=kudu/127.23.72.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:43907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:26:37.646726 11039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:26:37.646981 11039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:26:37.647022 11039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:26:37.651093 11039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:26:37.651146 11039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:26:37.651240 11039 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.129
I20260808 18:26:37.656401 11039 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:43907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab
--principal=kudu/127.23.72.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.72.129
--webserver_port=0
--tserver_master_addrs=127.23.72.190:40091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.129
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:26:37.657399 11039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:26:37.658456 11039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:26:37.670161 11045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:37.670195 11044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:37.670190 11048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:37.670516 11039 server_base.cc:1034] running on GCE node
I20260808 18:26:37.670872 11039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:26:37.671543 11039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:26:37.672732 11039 hybrid_clock.cc:648] HybridClock initialized: now 1786213597672716 us; error 26 us; skew 500 ppm
Aug 08 18:26:37 dist-test-slave-5k54 krb5kdc[10928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213597, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:26:37.680119 11039 init.cc:377] Logged in from keytab as kudu/127.23.72.129@KRBTEST.COM (short username kudu)
I20260808 18:26:37.680950 11039 webserver.cc:460] Webserver started at http://127.23.72.129:36353/ using document root <none> and password file <none>
I20260808 18:26:37.681474 11039 fs_manager.cc:359] Metadata directory not provided
I20260808 18:26:37.681530 11039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:26:37.681690 11039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:26:37.683759 11039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-0/data/instance:
uuid: "3fe06c4d1bba4d1ca624ec3cd1f8b79e"
format_stamp: "Formatted at 2026-08-08 18:26:37 on dist-test-slave-5k54"
server_key: "cd8235b26c656fe39cb9d37c9e5a9912"
server_key_iv: "b970a08899928e5bfcf5143a6e7ea349"
server_key_version: "encryptionkey@0"
I20260808 18:26:37.684345 11039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance:
uuid: "3fe06c4d1bba4d1ca624ec3cd1f8b79e"
format_stamp: "Formatted at 2026-08-08 18:26:37 on dist-test-slave-5k54"
server_key: "cd8235b26c656fe39cb9d37c9e5a9912"
server_key_iv: "b970a08899928e5bfcf5143a6e7ea349"
server_key_version: "encryptionkey@0"
I20260808 18:26:37.688154 11039 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260808 18:26:37.690522 11055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:37.691838 11039 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:26:37.691963 11039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-0/wal
uuid: "3fe06c4d1bba4d1ca624ec3cd1f8b79e"
format_stamp: "Formatted at 2026-08-08 18:26:37 on dist-test-slave-5k54"
server_key: "cd8235b26c656fe39cb9d37c9e5a9912"
server_key_iv: "b970a08899928e5bfcf5143a6e7ea349"
server_key_version: "encryptionkey@0"
I20260808 18:26:37.692072 11039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:26:37.709087 11039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:26:37.712080 11039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:26:37.712278 11039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:26:37.712910 11039 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:26:37.713747 11039 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:26:37.713799 11039 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:37.713868 11039 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:26:37.713898 11039 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:37.722994 11177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.129:41947 every 8 connection(s)
I20260808 18:26:37.722998 11039 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.129:41947
I20260808 18:26:37.724000 11039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
I20260808 18:26:37.731668 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 11039
I20260808 18:26:37.731814 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance
I20260808 18:26:37.732148 23842 external_mini_cluster.cc:1428] Setting key e7a81f98464f45c9b693f956b470b338
Aug 08 18:26:37 dist-test-slave-5k54 krb5kdc[10928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213597, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:26:37.738448 11015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:37.725774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:46799 (local address 127.23.72.190:40091)
0808 18:26:37.725957 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:37.725960 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:37.726749 (+ 789us) server_negotiation.cc:407] Connection header received
0808 18:26:37.728158 (+ 1409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:37.728161 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:37.728220 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:37.728329 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:37.729541 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:37.730070 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:37.731208 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:37.731512 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:37.731621 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:37.734622 (+ 3001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:37.734625 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:37.734627 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:26:37.734649 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:37.736324 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:37.736813 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:37.736818 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:37.736820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:37.736926 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:37.737255 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:37.737261 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:37.737266 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:37.737511 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:37.737620 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:37.738070 (+ 450us) server_negotiation.cc:299] Negotiation successful
0808 18:26:37.738210 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260808 18:26:37.739116 11180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:37.726184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:40091 (local address 127.23.72.129:46799)
0808 18:26:37.726615 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:37.726655 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:37.727949 (+ 1294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:37.728605 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:37.728612 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:37.728944 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:26:37.729399 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:37.729412 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:37.730198 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:37.730206 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:37.731084 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:37.731092 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:37.731659 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:37.731664 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:37.731871 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:37.732482 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:26:37.732498 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:26:37.734441 (+ 1943us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:26:37.736451 (+ 2010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:26:37.736455 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:26:37.736466 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:26:37.736716 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:26:37.737029 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:26:37.737032 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:26:37.737033 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:26:37.737161 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:26:37.737614 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:26:37.737619 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:26:37.737830 (+ 211us) client_negotiation.cc:769] Sending connection context
0808 18:26:37.737987 (+ 157us) client_negotiation.cc:240] Negotiation successful
0808 18:26:37.738219 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":121,"threads_started":1}
I20260808 18:26:37.740345 11178 heartbeater.cc:347] Connected to a master server at 127.23.72.190:40091
I20260808 18:26:37.740617 11178 heartbeater.cc:464] Registering TS with master...
I20260808 18:26:37.741215 11178 heartbeater.cc:511] Master 127.23.72.190:40091 requested a full tablet report, sending...
I20260808 18:26:37.742739 10975 ts_manager.cc:194] Registered new tserver with Master: 3fe06c4d1bba4d1ca624ec3cd1f8b79e (127.23.72.129:41947)
WARNING: no policy specified for kudu/127.23.72.130@KRBTEST.COM; defaulting to no policy
I20260808 18:26:37.744369 10975 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.129@KRBTEST.COM'} at 127.23.72.129:46799
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab.
Entry for principal kudu/127.23.72.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab.
I20260808 18:26:37.761936 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.130:0
--local_ip_for_outbound_sockets=127.23.72.130
--webserver_interface=127.23.72.130
--webserver_port=0
--tserver_master_addrs=127.23.72.190:40091
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab
--principal=kudu/127.23.72.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:43907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:26:37.876463 11183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:26:37.876852 11183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:26:37.876973 11183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:26:37.883534 11183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:26:37.883598 11183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:26:37.883697 11183 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.130
I20260808 18:26:37.891705 11183 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:43907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab
--principal=kudu/127.23.72.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.72.130
--webserver_port=0
--tserver_master_addrs=127.23.72.190:40091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11183
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.130
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:26:37.893268 11183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:26:37.894682 11183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:26:37.905047 11188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:37.905131 11183 server_base.cc:1034] running on GCE node
W20260808 18:26:37.905058 11191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:37.905267 11189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:37.905665 11183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:26:37.906333 11183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:26:37.907488 11183 hybrid_clock.cc:648] HybridClock initialized: now 1786213597907468 us; error 28 us; skew 500 ppm
Aug 08 18:26:37 dist-test-slave-5k54 krb5kdc[10928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213597, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:26:37.910171 11183 init.cc:377] Logged in from keytab as kudu/127.23.72.130@KRBTEST.COM (short username kudu)
I20260808 18:26:37.910974 11183 webserver.cc:460] Webserver started at http://127.23.72.130:39805/ using document root <none> and password file <none>
I20260808 18:26:37.911585 11183 fs_manager.cc:359] Metadata directory not provided
I20260808 18:26:37.911676 11183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:26:37.911890 11183 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:26:37.913889 11183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-1/data/instance:
uuid: "8804a273b96a424bbb625cf817e9798c"
format_stamp: "Formatted at 2026-08-08 18:26:37 on dist-test-slave-5k54"
server_key: "61da704cfeb4d466eb6697425044c1e1"
server_key_iv: "90585e620ea39576f632d7b04fcacdcd"
server_key_version: "encryptionkey@0"
I20260808 18:26:37.914463 11183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance:
uuid: "8804a273b96a424bbb625cf817e9798c"
format_stamp: "Formatted at 2026-08-08 18:26:37 on dist-test-slave-5k54"
server_key: "61da704cfeb4d466eb6697425044c1e1"
server_key_iv: "90585e620ea39576f632d7b04fcacdcd"
server_key_version: "encryptionkey@0"
I20260808 18:26:37.918099 11183 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260808 18:26:37.920521 11198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:37.921715 11183 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:26:37.921839 11183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-1/wal
uuid: "8804a273b96a424bbb625cf817e9798c"
format_stamp: "Formatted at 2026-08-08 18:26:37 on dist-test-slave-5k54"
server_key: "61da704cfeb4d466eb6697425044c1e1"
server_key_iv: "90585e620ea39576f632d7b04fcacdcd"
server_key_version: "encryptionkey@0"
I20260808 18:26:37.921954 11183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:26:37.938942 11183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:26:37.941891 11183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:26:37.942101 11183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:26:37.942668 11183 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:26:37.943557 11183 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:26:37.943621 11183 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:37.943688 11183 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:26:37.943737 11183 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:37.952448 11183 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.130:42747
I20260808 18:26:37.952517 11324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.130:42747 every 8 connection(s)
I20260808 18:26:37.953356 11183 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
I20260808 18:26:37.959705 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 11183
I20260808 18:26:37.959863 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance
I20260808 18:26:37.960196 23842 external_mini_cluster.cc:1428] Setting key 4bf05a66d49efe4cc14cbd687a6eebcb
Aug 08 18:26:37 dist-test-slave-5k54 krb5kdc[10928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213597, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.130@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:26:37.968120 11015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:37.955040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:46587 (local address 127.23.72.190:40091)
0808 18:26:37.955169 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:37.955173 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:37.955968 (+ 795us) server_negotiation.cc:407] Connection header received
0808 18:26:37.957334 (+ 1366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:37.957337 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:37.957396 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:37.957471 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:37.958550 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:37.959093 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:37.960706 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:37.960947 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:37.961060 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:37.963909 (+ 2849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:37.963915 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:37.963918 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:26:37.963947 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:37.965785 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:37.966479 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:37.966483 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:37.966484 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:37.966561 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:37.966973 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:37.966975 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:37.966977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:37.967141 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:37.967212 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:37.967907 (+ 695us) server_negotiation.cc:299] Negotiation successful
0808 18:26:37.968005 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 18:26:37.969379 11327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:37.955340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:40091 (local address 127.23.72.130:46587)
0808 18:26:37.955812 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:37.955858 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:37.957107 (+ 1249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:37.957629 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:37.957636 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:37.957959 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:26:37.958406 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:37.958418 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:37.959220 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:37.959224 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:37.960582 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:37.960592 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:37.961093 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:37.961097 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:37.961260 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:37.961855 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:26:37.961870 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:26:37.963689 (+ 1819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:26:37.965942 (+ 2253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:26:37.965951 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:26:37.965972 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:26:37.966350 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:26:37.966654 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:26:37.966660 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:26:37.966665 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:26:37.966857 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:26:37.967252 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:26:37.967262 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:26:37.967635 (+ 373us) client_negotiation.cc:769] Sending connection context
0808 18:26:37.967847 (+ 212us) client_negotiation.cc:240] Negotiation successful
0808 18:26:37.968220 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":118,"threads_started":1}
I20260808 18:26:37.970559 11325 heartbeater.cc:347] Connected to a master server at 127.23.72.190:40091
I20260808 18:26:37.970782 11325 heartbeater.cc:464] Registering TS with master...
I20260808 18:26:37.971251 11325 heartbeater.cc:511] Master 127.23.72.190:40091 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.23.72.131@KRBTEST.COM; defaulting to no policy
I20260808 18:26:37.972318 10975 ts_manager.cc:194] Registered new tserver with Master: 8804a273b96a424bbb625cf817e9798c (127.23.72.130:42747)
I20260808 18:26:37.972965 10975 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.130@KRBTEST.COM'} at 127.23.72.130:46587
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab.
Entry for principal kudu/127.23.72.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab.
I20260808 18:26:37.990404 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.131:0
--local_ip_for_outbound_sockets=127.23.72.131
--webserver_interface=127.23.72.131
--webserver_port=0
--tserver_master_addrs=127.23.72.190:40091
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab
--principal=kudu/127.23.72.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:43907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:26:38.100049 11330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:26:38.100323 11330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:26:38.100412 11330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:26:38.104369 11330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:26:38.104454 11330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:26:38.104544 11330 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.131
I20260808 18:26:38.109673 11330 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:43907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab
--principal=kudu/127.23.72.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.72.131
--webserver_port=0
--tserver_master_addrs=127.23.72.190:40091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.131
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:26:38.110720 11330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:26:38.111835 11330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:26:38.121397 11336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:38.121397 11338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:38.121415 11335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:38.121639 11330 server_base.cc:1034] running on GCE node
I20260808 18:26:38.122171 11330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:26:38.122776 11330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:26:38.123956 11330 hybrid_clock.cc:648] HybridClock initialized: now 1786213598123929 us; error 45 us; skew 500 ppm
Aug 08 18:26:38 dist-test-slave-5k54 krb5kdc[10928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213598, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:26:38.127055 11330 init.cc:377] Logged in from keytab as kudu/127.23.72.131@KRBTEST.COM (short username kudu)
I20260808 18:26:38.127965 11330 webserver.cc:460] Webserver started at http://127.23.72.131:42299/ using document root <none> and password file <none>
I20260808 18:26:38.128526 11330 fs_manager.cc:359] Metadata directory not provided
I20260808 18:26:38.128614 11330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:26:38.128856 11330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:26:38.130954 11330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-2/data/instance:
uuid: "3948ca27698e46449ea7ff036b5a57a4"
format_stamp: "Formatted at 2026-08-08 18:26:38 on dist-test-slave-5k54"
server_key: "f3a9743b0f55e5a9443721cec56aa412"
server_key_iv: "aebe8cf492c6e10152cefbc059ba8ba2"
server_key_version: "encryptionkey@0"
I20260808 18:26:38.131567 11330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance:
uuid: "3948ca27698e46449ea7ff036b5a57a4"
format_stamp: "Formatted at 2026-08-08 18:26:38 on dist-test-slave-5k54"
server_key: "f3a9743b0f55e5a9443721cec56aa412"
server_key_iv: "aebe8cf492c6e10152cefbc059ba8ba2"
server_key_version: "encryptionkey@0"
I20260808 18:26:38.135036 11330 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:26:38.137405 11345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:38.138666 11330 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 18:26:38.138803 11330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-2/wal
uuid: "3948ca27698e46449ea7ff036b5a57a4"
format_stamp: "Formatted at 2026-08-08 18:26:38 on dist-test-slave-5k54"
server_key: "f3a9743b0f55e5a9443721cec56aa412"
server_key_iv: "aebe8cf492c6e10152cefbc059ba8ba2"
server_key_version: "encryptionkey@0"
I20260808 18:26:38.138913 11330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:26:38.151512 11330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:26:38.154619 11330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:26:38.154843 11330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:26:38.155467 11330 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:26:38.156296 11330 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:26:38.156368 11330 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:38.156437 11330 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:26:38.156479 11330 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:38.165656 11330 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.131:34525
I20260808 18:26:38.165721 11464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.131:34525 every 8 connection(s)
I20260808 18:26:38.166584 11330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
I20260808 18:26:38.166932 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 11330
I20260808 18:26:38.167095 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance
I20260808 18:26:38.167495 23842 external_mini_cluster.cc:1428] Setting key d9835e11257fcf836e1d0be4ef408e38
Aug 08 18:26:38 dist-test-slave-5k54 krb5kdc[10928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213598, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.131@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:26:38.180039 11015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:38.168444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:45373 (local address 127.23.72.190:40091)
0808 18:26:38.168589 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:38.168593 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:38.169675 (+ 1082us) server_negotiation.cc:407] Connection header received
0808 18:26:38.171147 (+ 1472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:38.171150 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:38.171196 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:38.171263 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:38.172368 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:38.172887 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:38.173992 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:38.174200 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:38.174273 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:38.176751 (+ 2478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:38.176753 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:38.176755 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:26:38.176780 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:38.178280 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:38.178708 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:38.178711 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:38.178712 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:38.178784 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:38.179061 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:38.179063 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:38.179064 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:38.179227 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:38.179300 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:38.179801 (+ 501us) server_negotiation.cc:299] Negotiation successful
0808 18:26:38.179921 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 18:26:38.181011 11467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:38.168986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:40091 (local address 127.23.72.131:45373)
0808 18:26:38.169526 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:38.169582 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:38.170935 (+ 1353us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:38.171393 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:38.171399 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:38.171775 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:26:38.172241 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:38.172254 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:38.172991 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:38.172994 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:38.173884 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:38.173893 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:38.174324 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:38.174326 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:38.174476 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:38.175054 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:26:38.175069 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:26:38.176602 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:26:38.178373 (+ 1771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:26:38.178377 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:26:38.178387 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:26:38.178628 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:26:38.178859 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:26:38.178861 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:26:38.178862 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:26:38.178985 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:26:38.179311 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:26:38.179315 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:26:38.179601 (+ 286us) client_negotiation.cc:769] Sending connection context
0808 18:26:38.179752 (+ 151us) client_negotiation.cc:240] Negotiation successful
0808 18:26:38.179993 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":130,"threads_started":1}
I20260808 18:26:38.182094 11465 heartbeater.cc:347] Connected to a master server at 127.23.72.190:40091
I20260808 18:26:38.182329 11465 heartbeater.cc:464] Registering TS with master...
I20260808 18:26:38.182792 11465 heartbeater.cc:511] Master 127.23.72.190:40091 requested a full tablet report, sending...
I20260808 18:26:38.183820 10974 ts_manager.cc:194] Registered new tserver with Master: 3948ca27698e46449ea7ff036b5a57a4 (127.23.72.131:34525)
I20260808 18:26:38.184406 10974 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.131@KRBTEST.COM'} at 127.23.72.131:45373
I20260808 18:26:38.193454 23842 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:26:38.203127 11015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:38.195630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51360 (local address 127.23.72.190:40091)
0808 18:26:38.195788 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:38.195793 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:38.196093 (+ 300us) server_negotiation.cc:407] Connection header received
0808 18:26:38.196188 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:38.196190 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:38.196230 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:38.196291 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:38.196986 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:38.197492 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:38.198305 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:38.198537 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:38.198613 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:38.200256 (+ 1643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:38.200259 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:38.200261 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:26:38.200283 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:38.201506 (+ 1223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:38.201970 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:38.201972 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:38.201974 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:38.202046 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:38.202310 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:38.202312 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:38.202314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:38.202512 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:38.202624 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:38.202852 (+ 228us) server_negotiation.cc:299] Negotiation successful
0808 18:26:38.202954 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 18:26:38.206631 10974 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51360:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:26:38.209720 10974 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:26:38.224054 11476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:38.218104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.130:42747 (local address 127.0.0.1:51300)
0808 18:26:38.218613 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:38.218663 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:38.218809 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:38.219681 (+ 872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:38.219685 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:38.219706 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:26:38.219849 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:38.219856 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:38.221187 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:38.221193 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:38.222451 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:38.222463 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:38.223522 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:38.223526 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:38.223649 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:38.223692 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 18:26:38.223780 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 18:26:38.223888 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"mutex_wait_us":74,"thread_start_us":96,"threads_started":1}
I20260808 18:26:38.224701 11477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:38.218276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51300 (local address 127.23.72.130:42747)
0808 18:26:38.218714 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:38.218729 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:38.218756 (+ 27us) server_negotiation.cc:407] Connection header received
0808 18:26:38.219041 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:38.219047 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:38.219263 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:38.219478 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:26:38.219984 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:38.221065 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:38.222587 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:38.223345 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:38.224267 (+ 922us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:38.224422 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:38.224475 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 18:26:38.224562 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":127,"threads_started":1}
I20260808 18:26:38.225375 11479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:38.218841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:41947 (local address 127.0.0.1:34996)
0808 18:26:38.220107 (+ 1266us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:38.220120 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:38.220216 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:38.220739 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:38.220742 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:38.220756 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:26:38.220833 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:38.220837 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:38.222578 (+ 1741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:38.222581 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:38.223716 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:38.223725 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:38.224900 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:38.224903 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:38.225081 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:38.225106 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 18:26:38.225176 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 18:26:38.225243 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1208,"thread_start_us":97,"threads_started":1}
I20260808 18:26:38.225678 11478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:38.218289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.131:34525 (local address 127.0.0.1:56920)
0808 18:26:38.219883 (+ 1594us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:38.219894 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:38.220008 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:38.220504 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:38.220507 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:38.220525 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:26:38.220609 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:38.220615 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:38.221995 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:38.221998 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:38.223298 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:38.223309 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:38.224629 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:38.224633 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:38.224729 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:38.224771 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 18:26:38.224849 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 18:26:38.225556 (+ 707us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1538,"mutex_wait_us":2,"thread_start_us":114,"threads_started":1}
I20260808 18:26:38.225781 11480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:38.219029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34996 (local address 127.23.72.129:41947)
0808 18:26:38.219652 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:38.219671 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:38.220216 (+ 545us) server_negotiation.cc:407] Connection header received
0808 18:26:38.220317 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:38.220321 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:38.220490 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:38.220616 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:26:38.220948 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:38.222424 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:38.223827 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:38.224682 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:38.224830 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:38.224969 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:38.225542 (+ 573us) server_negotiation.cc:299] Negotiation successful
0808 18:26:38.225642 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":80,"threads_started":1}
I20260808 18:26:38.226594 11481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:38.219144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56920 (local address 127.23.72.131:34525)
0808 18:26:38.219569 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:38.219581 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:38.219987 (+ 406us) server_negotiation.cc:407] Connection header received
0808 18:26:38.220147 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:38.220151 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:38.220365 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:38.220513 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:26:38.220707 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:38.221841 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:38.223465 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:38.224334 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:38.224451 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:38.224567 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:38.225114 (+ 547us) server_negotiation.cc:299] Negotiation successful
0808 18:26:38.225784 (+ 670us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":158,"threads_started":1}
I20260808 18:26:38.227823 11250 tablet_service.cc:1467] Processing CreateTablet for tablet ab07a691ecca4e7bae4d56f5f78cfe36 (DEFAULT_TABLE table=test-table [id=2506c12fd55d48dbaafbeca89630d336]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:26:38.228086 11104 tablet_service.cc:1467] Processing CreateTablet for tablet ab07a691ecca4e7bae4d56f5f78cfe36 (DEFAULT_TABLE table=test-table [id=2506c12fd55d48dbaafbeca89630d336]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:26:38.228895 11104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab07a691ecca4e7bae4d56f5f78cfe36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:26:38.228940 11250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab07a691ecca4e7bae4d56f5f78cfe36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:26:38.229059 11393 tablet_service.cc:1467] Processing CreateTablet for tablet ab07a691ecca4e7bae4d56f5f78cfe36 (DEFAULT_TABLE table=test-table [id=2506c12fd55d48dbaafbeca89630d336]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:26:38.230113 11393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab07a691ecca4e7bae4d56f5f78cfe36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:26:38.234788 11482 tablet_bootstrap.cc:492] T ab07a691ecca4e7bae4d56f5f78cfe36 P 3fe06c4d1bba4d1ca624ec3cd1f8b79e: Bootstrap starting.
I20260808 18:26:38.236665 11483 tablet_bootstrap.cc:492] T ab07a691ecca4e7bae4d56f5f78cfe36 P 3948ca27698e46449ea7ff036b5a57a4: Bootstrap starting.
I20260808 18:26:38.237519 11484 tablet_bootstrap.cc:492] T ab07a691ecca4e7bae4d56f5f78cfe36 P 8804a273b96a424bbb625cf817e9798c: Bootstrap starting.
I20260808 18:26:38.237840 11482 tablet_bootstrap.cc:654] T ab07a691ecca4e7bae4d56f5f78cfe36 P 3fe06c4d1bba4d1ca624ec3cd1f8b79e: Neither blocks nor log segments found. Creating new log.
I20260808 18:26:38.238540 11483 tablet_bootstrap.cc:654] T ab07a691ecca4e7bae4d56f5f78cfe36 P 3948ca27698e46449ea7ff036b5a57a4: Neither blocks nor log segments found. Creating new log.
I20260808 18:26:38.239207 11482 log.cc:824] T ab07a691ecca4e7bae4d56f5f78cfe36 P 3fe06c4d1bba4d1ca624ec3cd1f8b79e: Log is configured to *not* fsync() on all Append() calls
I20260808 18:26:38.239768 11484 tablet_bootstrap.cc:654] T ab07a691ecca4e7bae4d56f5f78cfe36 P 8804a273b96a424bbb625cf817e9798c: Neither blocks nor log segments found. Creating new log.
I20260808 18:26:38.239759 11483 log.cc:824] T ab07a691ecca4e7bae4d56f5f78cfe36 P 3948ca27698e46449ea7ff036b5a57a4: Log is configured to *not* fsync() on all Append() calls
I20260808 18:26:38.241016 11484 log.cc:824] T ab07a691ecca4e7bae4d56f5f78cfe36 P 8804a273b96a424bbb625cf817e9798c: Log is configured to *not* fsync() on all Append() calls
I20260808 18:26:38.241453 11482 tablet_bootstrap.cc:492] T ab07a691ecca4e7bae4d56f5f78cfe36 P 3fe06c4d1bba4d1ca624ec3cd1f8b79e: No bootstrap required, opened a new log
I20260808 18:26:38.241684 11482 ts_tablet_manager.cc:1397] T ab07a691ecca4e7bae4d56f5f78cfe36 P 3fe06c4d1bba4d1ca624ec3cd1f8b79e: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 18:26:38.242113 11483 tablet_bootstrap.cc:492] T ab07a691ecca4e7bae4d56f5f78cfe36 P 3948ca27698e46449ea7ff036b5a57a4: No bootstrap required, opened a new log
I20260808 18:26:38.242321 11483 ts_tablet_manager.cc:1397] T ab07a691ecca4e7bae4d56f5f78cfe36 P 3948ca27698e46449ea7ff036b5a57a4: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 18:26:38.242983 11484 tablet_bootstrap.cc:492] T ab07a691ecca4e7bae4d56f5f78cfe36 P 8804a273b96a424bbb625cf817e9798c: No bootstrap required, opened a new log
I20260808 18:26:38.243157 11484 ts_tablet_manager.cc:1397] T ab07a691ecca4e7bae4d56f5f78cfe36 P 8804a273b96a424bbb625cf817e9798c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 18:26:38.245364 11483 raft_consensus.cc:348] T ab07a691ecca4e7bae4d56f5f78cfe36 P 3948ca27698e46449ea7ff036b5a57a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8804a273b96a424bbb625cf817e9798c" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 42747 } } peers { permanent_uuid: "3948ca27698e46449ea7ff036b5a57a4" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 34525 } } peers { permanent_uuid: "3fe06c4d1bba4d1ca624ec3cd1f8b79e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41947 } }
I20260808 18:26:38.245743 11483 raft_consensus.cc:374] T ab07a691ecca4e7bae4d56f5f78cfe36 P 3948ca27698e46449ea7ff036b5a57a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:26:38.245596 11482 raft_consensus.cc:348] T ab07a691ecca4e7bae4d56f5f78cfe36 P 3fe06c4d1bba4d1ca624ec3cd1f8b79e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8804a273b96a424bbb625cf817e9798c" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 42747 } } peers { permanent_uuid: "3948ca27698e46449ea7ff036b5a57a4" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 34525 } } peers { permanent_uuid: "3fe06c4d1bba4d1ca624ec3cd1f8b79e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41947 } }
I20260808 18:26:38.245815 11483 raft_consensus.cc:729] T ab07a691ecca4e7bae4d56f5f78cfe36 P 3948ca27698e46449ea7ff036b5a57a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3948ca27698e46449ea7ff036b5a57a4, State: Initialized, Role: FOLLOWER
I20260808 18:26:38.245834 11482 raft_consensus.cc:374] T ab07a691ecca4e7bae4d56f5f78cfe36 P 3fe06c4d1bba4d1ca624ec3cd1f8b79e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:26:38.245882 11482 raft_consensus.cc:729] T ab07a691ecca4e7bae4d56f5f78cfe36 P 3fe06c4d1bba4d1ca624ec3cd1f8b79e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fe06c4d1bba4d1ca624ec3cd1f8b79e, State: Initialized, Role: FOLLOWER
I20260808 18:26:38.246282 11483 consensus_queue.cc:260] T ab07a691ecca4e7bae4d56f5f78cfe36 P 3948ca27698e46449ea7ff036b5a57a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8804a273b96a424bbb625cf817e9798c" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 42747 } } peers { permanent_uuid: "3948ca27698e46449ea7ff036b5a57a4" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 34525 } } peers { permanent_uuid: "3fe06c4d1bba4d1ca624ec3cd1f8b79e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41947 } }
I20260808 18:26:38.246335 11482 consensus_queue.cc:260] T ab07a691ecca4e7bae4d56f5f78cfe36 P 3fe06c4d1bba4d1ca624ec3cd1f8b79e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8804a273b96a424bbb625cf817e9798c" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 42747 } } peers { permanent_uuid: "3948ca27698e46449ea7ff036b5a57a4" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 34525 } } peers { permanent_uuid: "3fe06c4d1bba4d1ca624ec3cd1f8b79e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41947 } }
I20260808 18:26:38.246852 11484 raft_consensus.cc:348] T ab07a691ecca4e7bae4d56f5f78cfe36 P 8804a273b96a424bbb625cf817e9798c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8804a273b96a424bbb625cf817e9798c" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 42747 } } peers { permanent_uuid: "3948ca27698e46449ea7ff036b5a57a4" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 34525 } } peers { permanent_uuid: "3fe06c4d1bba4d1ca624ec3cd1f8b79e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41947 } }
I20260808 18:26:38.247102 11484 raft_consensus.cc:374] T ab07a691ecca4e7bae4d56f5f78cfe36 P 8804a273b96a424bbb625cf817e9798c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:26:38.247164 11484 raft_consensus.cc:729] T ab07a691ecca4e7bae4d56f5f78cfe36 P 8804a273b96a424bbb625cf817e9798c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8804a273b96a424bbb625cf817e9798c, State: Initialized, Role: FOLLOWER
I20260808 18:26:38.247174 11465 heartbeater.cc:503] Master 127.23.72.190:40091 was elected leader, sending a full tablet report...
I20260808 18:26:38.247308 11482 ts_tablet_manager.cc:1428] T ab07a691ecca4e7bae4d56f5f78cfe36 P 3fe06c4d1bba4d1ca624ec3cd1f8b79e: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 18:26:38.247598 11484 consensus_queue.cc:260] T ab07a691ecca4e7bae4d56f5f78cfe36 P 8804a273b96a424bbb625cf817e9798c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8804a273b96a424bbb625cf817e9798c" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 42747 } } peers { permanent_uuid: "3948ca27698e46449ea7ff036b5a57a4" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 34525 } } peers { permanent_uuid: "3fe06c4d1bba4d1ca624ec3cd1f8b79e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41947 } }
I20260808 18:26:38.247697 11178 heartbeater.cc:503] Master 127.23.72.190:40091 was elected leader, sending a full tablet report...
I20260808 18:26:38.248235 11325 heartbeater.cc:503] Master 127.23.72.190:40091 was elected leader, sending a full tablet report...
I20260808 18:26:38.248490 11484 ts_tablet_manager.cc:1428] T ab07a691ecca4e7bae4d56f5f78cfe36 P 8804a273b96a424bbb625cf817e9798c: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 18:26:38.248858 11483 ts_tablet_manager.cc:1428] T ab07a691ecca4e7bae4d56f5f78cfe36 P 3948ca27698e46449ea7ff036b5a57a4: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 18:26:38.399087 11489 raft_consensus.cc:482] T ab07a691ecca4e7bae4d56f5f78cfe36 P 3fe06c4d1bba4d1ca624ec3cd1f8b79e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:26:38.399343 11489 raft_consensus.cc:504] T ab07a691ecca4e7bae4d56f5f78cfe36 P 3fe06c4d1bba4d1ca624ec3cd1f8b79e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8804a273b96a424bbb625cf817e9798c" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 42747 } } peers { permanent_uuid: "3948ca27698e46449ea7ff036b5a57a4" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 34525 } } peers { permanent_uuid: "3fe06c4d1bba4d1ca624ec3cd1f8b79e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41947 } }
I20260808 18:26:38.400772 11489 leader_election.cc:288] T ab07a691ecca4e7bae4d56f5f78cfe36 P 3fe06c4d1bba4d1ca624ec3cd1f8b79e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8804a273b96a424bbb625cf817e9798c (127.23.72.130:42747), 3948ca27698e46449ea7ff036b5a57a4 (127.23.72.131:34525)
I20260808 18:26:38.405161 11481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:38.401158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:35093 (local address 127.23.72.131:34525)
0808 18:26:38.401311 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:38.401316 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:38.401708 (+ 392us) server_negotiation.cc:407] Connection header received
0808 18:26:38.401847 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:38.401849 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:38.401892 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:38.402006 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:26:38.402412 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:38.402889 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:38.404215 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:38.404603 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:38.404698 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:38.404738 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:38.404997 (+ 259us) server_negotiation.cc:299] Negotiation successful
0808 18:26:38.405057 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 18:26:38.405225 11492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:38.401069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.131:34525 (local address 127.23.72.129:35093)
0808 18:26:38.401605 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:38.401622 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:38.401739 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:38.402054 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:38.402058 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:38.402081 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:26:38.402283 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:38.402291 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:38.403015 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:38.403020 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:38.404057 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:38.404068 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:38.404768 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:38.404772 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:38.404873 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:38.404899 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 18:26:38.404975 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 18:26:38.405041 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"mutex_wait_us":66,"thread_start_us":133,"threads_started":1}
I20260808 18:26:38.405903 11477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:38.401127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:57497 (local address 127.23.72.130:42747)
0808 18:26:38.401302 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:38.401308 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:38.401590 (+ 282us) server_negotiation.cc:407] Connection header received
0808 18:26:38.401700 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:38.401705 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:38.401837 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:38.401933 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:26:38.402295 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:38.402906 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:38.404181 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:38.404683 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:38.405485 (+ 802us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:38.405539 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:38.405725 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 18:26:38.405798 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 18:26:38.405901 11493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:38.401038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.130:42747 (local address 127.23.72.129:57497)
0808 18:26:38.401475 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:38.401488 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:38.401602 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:38.401957 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:38.401963 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:38.401986 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:26:38.402165 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:38.402171 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:38.403054 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:38.403057 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:38.404038 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:38.404048 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:38.405476 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:38.405479 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:38.405585 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:38.405603 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 18:26:38.405696 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 18:26:38.405765 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"mutex_wait_us":60,"thread_start_us":206,"threads_started":1}
I20260808 18:26:38.405978 11416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab07a691ecca4e7bae4d56f5f78cfe36" candidate_uuid: "3fe06c4d1bba4d1ca624ec3cd1f8b79e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3948ca27698e46449ea7ff036b5a57a4" is_pre_election: true
I20260808 18:26:38.406406 11416 raft_consensus.cc:2454] T ab07a691ecca4e7bae4d56f5f78cfe36 P 3948ca27698e46449ea7ff036b5a57a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fe06c4d1bba4d1ca624ec3cd1f8b79e in term 0.
I20260808 18:26:38.406692 11270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab07a691ecca4e7bae4d56f5f78cfe36" candidate_uuid: "3fe06c4d1bba4d1ca624ec3cd1f8b79e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8804a273b96a424bbb625cf817e9798c" is_pre_election: true
I20260808 18:26:38.406970 11270 raft_consensus.cc:2454] T ab07a691ecca4e7bae4d56f5f78cfe36 P 8804a273b96a424bbb625cf817e9798c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fe06c4d1bba4d1ca624ec3cd1f8b79e in term 0.
I20260808 18:26:38.406960 11057 leader_election.cc:302] T ab07a691ecca4e7bae4d56f5f78cfe36 P 3fe06c4d1bba4d1ca624ec3cd1f8b79e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3948ca27698e46449ea7ff036b5a57a4, 3fe06c4d1bba4d1ca624ec3cd1f8b79e; no voters:
I20260808 18:26:38.407318 11489 raft_consensus.cc:2781] T ab07a691ecca4e7bae4d56f5f78cfe36 P 3fe06c4d1bba4d1ca624ec3cd1f8b79e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:26:38.407454 11489 raft_consensus.cc:482] T ab07a691ecca4e7bae4d56f5f78cfe36 P 3fe06c4d1bba4d1ca624ec3cd1f8b79e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:26:38.407516 11489 raft_consensus.cc:3037] T ab07a691ecca4e7bae4d56f5f78cfe36 P 3fe06c4d1bba4d1ca624ec3cd1f8b79e [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:26:38.408594 11489 raft_consensus.cc:504] T ab07a691ecca4e7bae4d56f5f78cfe36 P 3fe06c4d1bba4d1ca624ec3cd1f8b79e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8804a273b96a424bbb625cf817e9798c" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 42747 } } peers { permanent_uuid: "3948ca27698e46449ea7ff036b5a57a4" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 34525 } } peers { permanent_uuid: "3fe06c4d1bba4d1ca624ec3cd1f8b79e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41947 } }
I20260808 18:26:38.408948 11489 leader_election.cc:288] T ab07a691ecca4e7bae4d56f5f78cfe36 P 3fe06c4d1bba4d1ca624ec3cd1f8b79e [CANDIDATE]: Term 1 election: Requested vote from peers 8804a273b96a424bbb625cf817e9798c (127.23.72.130:42747), 3948ca27698e46449ea7ff036b5a57a4 (127.23.72.131:34525)
I20260808 18:26:38.409289 11270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab07a691ecca4e7bae4d56f5f78cfe36" candidate_uuid: "3fe06c4d1bba4d1ca624ec3cd1f8b79e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8804a273b96a424bbb625cf817e9798c"
I20260808 18:26:38.409364 11416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab07a691ecca4e7bae4d56f5f78cfe36" candidate_uuid: "3fe06c4d1bba4d1ca624ec3cd1f8b79e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3948ca27698e46449ea7ff036b5a57a4"
I20260808 18:26:38.409402 11270 raft_consensus.cc:3037] T ab07a691ecca4e7bae4d56f5f78cfe36 P 8804a273b96a424bbb625cf817e9798c [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:26:38.409524 11416 raft_consensus.cc:3037] T ab07a691ecca4e7bae4d56f5f78cfe36 P 3948ca27698e46449ea7ff036b5a57a4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:26:38.410522 11270 raft_consensus.cc:2454] T ab07a691ecca4e7bae4d56f5f78cfe36 P 8804a273b96a424bbb625cf817e9798c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fe06c4d1bba4d1ca624ec3cd1f8b79e in term 1.
I20260808 18:26:38.410837 11059 leader_election.cc:302] T ab07a691ecca4e7bae4d56f5f78cfe36 P 3fe06c4d1bba4d1ca624ec3cd1f8b79e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fe06c4d1bba4d1ca624ec3cd1f8b79e, 8804a273b96a424bbb625cf817e9798c; no voters:
I20260808 18:26:38.410951 11416 raft_consensus.cc:2454] T ab07a691ecca4e7bae4d56f5f78cfe36 P 3948ca27698e46449ea7ff036b5a57a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fe06c4d1bba4d1ca624ec3cd1f8b79e in term 1.
I20260808 18:26:38.411005 11489 raft_consensus.cc:2781] T ab07a691ecca4e7bae4d56f5f78cfe36 P 3fe06c4d1bba4d1ca624ec3cd1f8b79e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:26:38.411273 11489 raft_consensus.cc:686] T ab07a691ecca4e7bae4d56f5f78cfe36 P 3fe06c4d1bba4d1ca624ec3cd1f8b79e [term 1 LEADER]: Becoming Leader. State: Replica: 3fe06c4d1bba4d1ca624ec3cd1f8b79e, State: Running, Role: LEADER
I20260808 18:26:38.411643 11489 consensus_queue.cc:237] T ab07a691ecca4e7bae4d56f5f78cfe36 P 3fe06c4d1bba4d1ca624ec3cd1f8b79e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8804a273b96a424bbb625cf817e9798c" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 42747 } } peers { permanent_uuid: "3948ca27698e46449ea7ff036b5a57a4" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 34525 } } peers { permanent_uuid: "3fe06c4d1bba4d1ca624ec3cd1f8b79e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41947 } }
I20260808 18:26:38.414991 10975 catalog_manager.cc:5495] T ab07a691ecca4e7bae4d56f5f78cfe36 P 3fe06c4d1bba4d1ca624ec3cd1f8b79e reported cstate change: term changed from 0 to 1, leader changed from <none> to 3fe06c4d1bba4d1ca624ec3cd1f8b79e (127.23.72.129). New cstate: current_term: 1 leader_uuid: "3fe06c4d1bba4d1ca624ec3cd1f8b79e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8804a273b96a424bbb625cf817e9798c" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 42747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3948ca27698e46449ea7ff036b5a57a4" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 34525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fe06c4d1bba4d1ca624ec3cd1f8b79e" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 41947 } health_report { overall_health: HEALTHY } } }
W20260808 18:26:38.417788 11466 tablet.cc:2369] T ab07a691ecca4e7bae4d56f5f78cfe36 P 3948ca27698e46449ea7ff036b5a57a4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 18:26:38.454458 11326 tablet.cc:2369] T ab07a691ecca4e7bae4d56f5f78cfe36 P 8804a273b96a424bbb625cf817e9798c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:26:38.466065 11480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:38.462502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35006 (local address 127.23.72.129:41947)
0808 18:26:38.462622 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:38.462625 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:38.462641 (+ 16us) server_negotiation.cc:407] Connection header received
0808 18:26:38.462730 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:38.462733 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:38.462787 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:38.462882 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:26:38.463120 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:38.463761 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:38.464572 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:38.464835 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:38.464922 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:38.465270 (+ 348us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:26:38.465656 (+ 386us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:26:38.465762 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:38.465849 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 18:26:38.465884 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 18:26:38.471788 11270 raft_consensus.cc:1261] T ab07a691ecca4e7bae4d56f5f78cfe36 P 8804a273b96a424bbb625cf817e9798c [term 1 FOLLOWER]: Refusing update from remote peer 3fe06c4d1bba4d1ca624ec3cd1f8b79e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:26:38.471788 11416 raft_consensus.cc:1261] T ab07a691ecca4e7bae4d56f5f78cfe36 P 3948ca27698e46449ea7ff036b5a57a4 [term 1 FOLLOWER]: Refusing update from remote peer 3fe06c4d1bba4d1ca624ec3cd1f8b79e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:26:38.472503 11489 consensus_queue.cc:1035] T ab07a691ecca4e7bae4d56f5f78cfe36 P 3fe06c4d1bba4d1ca624ec3cd1f8b79e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3948ca27698e46449ea7ff036b5a57a4" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 34525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:26:38.472688 11494 consensus_queue.cc:1035] T ab07a691ecca4e7bae4d56f5f78cfe36 P 3fe06c4d1bba4d1ca624ec3cd1f8b79e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8804a273b96a424bbb625cf817e9798c" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 42747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 18:26:38.475229 11179 tablet.cc:2369] T ab07a691ecca4e7bae4d56f5f78cfe36 P 3fe06c4d1bba4d1ca624ec3cd1f8b79e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:26:38.480315 11499 mvcc.cc:205] Tried to move back new op lower bound from 7316330899332792320 to 7316330899096678400. Current Snapshot: MvccSnapshot[applied={T|T < 7316330899332792320}]
I20260808 18:26:38.480871 11498 mvcc.cc:205] Tried to move back new op lower bound from 7316330899332792320 to 7316330899096678400. Current Snapshot: MvccSnapshot[applied={T|T < 7316330899332792320}]
I20260808 18:26:38.488654 10973 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51360:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:26:38.488837 10973 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51360:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:26:38.491374 10973 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2f667c524c63416cafbb443054a412b5: Sending DeleteTablet for 3 replicas of tablet ab07a691ecca4e7bae4d56f5f78cfe36
I20260808 18:26:38.492146 11250 tablet_service.cc:1514] Processing DeleteTablet for tablet ab07a691ecca4e7bae4d56f5f78cfe36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:26:38 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:51300
I20260808 18:26:38.492244 11393 tablet_service.cc:1514] Processing DeleteTablet for tablet ab07a691ecca4e7bae4d56f5f78cfe36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:26:38 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:56920
I20260808 18:26:38.492637 11506 tablet_replica.cc:330] T ab07a691ecca4e7bae4d56f5f78cfe36 P 8804a273b96a424bbb625cf817e9798c: stopping tablet replica
I20260808 18:26:38.492944 11104 tablet_service.cc:1514] Processing DeleteTablet for tablet ab07a691ecca4e7bae4d56f5f78cfe36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:26:38 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:34996
I20260808 18:26:38.493001 11506 raft_consensus.cc:2229] T ab07a691ecca4e7bae4d56f5f78cfe36 P 8804a273b96a424bbb625cf817e9798c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:26:38.493131 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 11039
I20260808 18:26:38.493284 11506 raft_consensus.cc:2258] T ab07a691ecca4e7bae4d56f5f78cfe36 P 8804a273b96a424bbb625cf817e9798c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:26:38.496047 11507 tablet_replica.cc:330] T ab07a691ecca4e7bae4d56f5f78cfe36 P 3948ca27698e46449ea7ff036b5a57a4: stopping tablet replica
I20260808 18:26:38.496400 11507 raft_consensus.cc:2229] T ab07a691ecca4e7bae4d56f5f78cfe36 P 3948ca27698e46449ea7ff036b5a57a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:26:38.496697 11507 raft_consensus.cc:2258] T ab07a691ecca4e7bae4d56f5f78cfe36 P 3948ca27698e46449ea7ff036b5a57a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:26:38.497080 11506 ts_tablet_manager.cc:1905] T ab07a691ecca4e7bae4d56f5f78cfe36 P 8804a273b96a424bbb625cf817e9798c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:26:38.497886 11507 ts_tablet_manager.cc:1905] T ab07a691ecca4e7bae4d56f5f78cfe36 P 3948ca27698e46449ea7ff036b5a57a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:26:38.499958 11507 ts_tablet_manager.cc:1918] T ab07a691ecca4e7bae4d56f5f78cfe36 P 3948ca27698e46449ea7ff036b5a57a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:26:38.500072 11507 log.cc:1196] T ab07a691ecca4e7bae4d56f5f78cfe36 P 3948ca27698e46449ea7ff036b5a57a4: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-2/wal/wals/ab07a691ecca4e7bae4d56f5f78cfe36
I20260808 18:26:38.500375 11507 ts_tablet_manager.cc:1939] T ab07a691ecca4e7bae4d56f5f78cfe36 P 3948ca27698e46449ea7ff036b5a57a4: Deleting consensus metadata
I20260808 18:26:38.500579 11506 ts_tablet_manager.cc:1918] T ab07a691ecca4e7bae4d56f5f78cfe36 P 8804a273b96a424bbb625cf817e9798c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:26:38.500689 11506 log.cc:1196] T ab07a691ecca4e7bae4d56f5f78cfe36 P 8804a273b96a424bbb625cf817e9798c: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213345431840-23842-0/minicluster-data/ts-1/wal/wals/ab07a691ecca4e7bae4d56f5f78cfe36
I20260808 18:26:38.501012 11506 ts_tablet_manager.cc:1939] T ab07a691ecca4e7bae4d56f5f78cfe36 P 8804a273b96a424bbb625cf817e9798c: Deleting consensus metadata
I20260808 18:26:38.501540 10960 catalog_manager.cc:4841] TS 3948ca27698e46449ea7ff036b5a57a4 (127.23.72.131:34525): tablet ab07a691ecca4e7bae4d56f5f78cfe36 (table test-table [id=2506c12fd55d48dbaafbeca89630d336]) successfully deleted
I20260808 18:26:38.502197 10962 catalog_manager.cc:4841] TS 8804a273b96a424bbb625cf817e9798c (127.23.72.130:42747): tablet ab07a691ecca4e7bae4d56f5f78cfe36 (table test-table [id=2506c12fd55d48dbaafbeca89630d336]) successfully deleted
W20260808 18:26:38.502810 10960 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.72.129:41947 (error 108)
I20260808 18:26:38.503674 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 11183
I20260808 18:26:38.503911 11478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:38.503575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:41947 (local address 127.0.0.1:35016)
0808 18:26:38.503727 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:38.503808 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.129:41947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260808 18:26:38.505146 10960 catalog_manager.cc:4584] TS 3fe06c4d1bba4d1ca624ec3cd1f8b79e (127.23.72.129:41947): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ab07a691ecca4e7bae4d56f5f78cfe36: Network error: Client connection negotiation failed: client connection to 127.23.72.129:41947: connect: Connection refused (error 111)
I20260808 18:26:38.511554 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 11330
I20260808 18:26:38.518370 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 10941
2026-08-08T18:26:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2459 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-08T18:26:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:26:38Z Disabled control of system clock
I20260808 18:26:38.551533 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:33475
--webserver_interface=127.23.72.190
--webserver_port=0
--builtin_ntp_servers=127.23.72.148:33059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:33475
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 18:26:38.661749 11513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:26:38.662065 11513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:26:38.662101 11513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:26:38.666111 11513 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:26:38.666165 11513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:26:38.666181 11513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:26:38.666198 11513 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:26:38.666211 11513 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:26:38.672340 11513 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:33059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:33475
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:33475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.72.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11513
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:26:38.673404 11513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:26:38.674798 11513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:26:38.683398 11518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:38.683496 11513 server_base.cc:1034] running on GCE node
W20260808 18:26:38.683377 11522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:38.683393 11520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:38.684345 11513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:26:38.685489 11513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:26:38.686676 11513 hybrid_clock.cc:648] HybridClock initialized: now 1786213598686647 us; error 43 us; skew 500 ppm
I20260808 18:26:38.688508 11513 webserver.cc:460] Webserver started at http://127.23.72.190:34927/ using document root <none> and password file <none>
I20260808 18:26:38.689077 11513 fs_manager.cc:359] Metadata directory not provided
I20260808 18:26:38.689167 11513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:26:38.689399 11513 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:26:38.691395 11513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "7ad50cbc15e44d428f073d146c382b4e"
format_stamp: "Formatted at 2026-08-08 18:26:38 on dist-test-slave-5k54"
server_key: "b24d70387e62765a5abf00b5f3823f6f"
server_key_iv: "1358578e2beb2a06522d0f4a07992584"
server_key_version: "encryptionkey@0"
I20260808 18:26:38.692029 11513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "7ad50cbc15e44d428f073d146c382b4e"
format_stamp: "Formatted at 2026-08-08 18:26:38 on dist-test-slave-5k54"
server_key: "b24d70387e62765a5abf00b5f3823f6f"
server_key_iv: "1358578e2beb2a06522d0f4a07992584"
server_key_version: "encryptionkey@0"
I20260808 18:26:38.695582 11513 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 18:26:38.697818 11529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:38.699004 11513 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:26:38.699152 11513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "7ad50cbc15e44d428f073d146c382b4e"
format_stamp: "Formatted at 2026-08-08 18:26:38 on dist-test-slave-5k54"
server_key: "b24d70387e62765a5abf00b5f3823f6f"
server_key_iv: "1358578e2beb2a06522d0f4a07992584"
server_key_version: "encryptionkey@0"
I20260808 18:26:38.699257 11513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:26:38.719345 11513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:26:38.720059 11513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:26:38.720264 11513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:26:38.727471 11513 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.190:33475
I20260808 18:26:38.727492 11586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.190:33475 every 8 connection(s)
I20260808 18:26:38.728395 11513 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:26:38.731482 11587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:26:38.736760 11587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ad50cbc15e44d428f073d146c382b4e: Bootstrap starting.
I20260808 18:26:38.737471 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 11513
I20260808 18:26:38.737593 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:26:38.737885 23842 external_mini_cluster.cc:1428] Setting key 98675a1254485c7070952a9fd9a81545
I20260808 18:26:38.739588 11587 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ad50cbc15e44d428f073d146c382b4e: Neither blocks nor log segments found. Creating new log.
I20260808 18:26:38.740476 11587 log.cc:824] T 00000000000000000000000000000000 P 7ad50cbc15e44d428f073d146c382b4e: Log is configured to *not* fsync() on all Append() calls
I20260808 18:26:38.742653 11587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ad50cbc15e44d428f073d146c382b4e: No bootstrap required, opened a new log
I20260808 18:26:38.747031 11587 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ad50cbc15e44d428f073d146c382b4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ad50cbc15e44d428f073d146c382b4e" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 33475 } }
I20260808 18:26:38.747287 11587 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ad50cbc15e44d428f073d146c382b4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:26:38.747344 11587 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ad50cbc15e44d428f073d146c382b4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ad50cbc15e44d428f073d146c382b4e, State: Initialized, Role: FOLLOWER
I20260808 18:26:38.747814 11587 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ad50cbc15e44d428f073d146c382b4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ad50cbc15e44d428f073d146c382b4e" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 33475 } }
I20260808 18:26:38.747982 11587 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ad50cbc15e44d428f073d146c382b4e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:26:38.748046 11587 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ad50cbc15e44d428f073d146c382b4e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:26:38.748140 11587 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ad50cbc15e44d428f073d146c382b4e [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:26:38.748204 11590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:38.739108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46716 (local address 127.23.72.190:33475)
0808 18:26:38.739906 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:38.739925 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:38.739967 (+ 42us) server_negotiation.cc:407] Connection header received
0808 18:26:38.740816 (+ 849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:38.740839 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:38.741185 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:38.741569 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:38.742040 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:38.743496 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:38.744420 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:38.745147 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:38.745304 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:38.746165 (+ 861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:38.746175 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:38.746194 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:26:38.746696 (+ 502us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:38.746766 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:26:38.746790 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:38.746889 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:38.746985 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 18:26:38.747241 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":131,"threads_started":1}
I20260808 18:26:38.749020 11587 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ad50cbc15e44d428f073d146c382b4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ad50cbc15e44d428f073d146c382b4e" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 33475 } }
I20260808 18:26:38.749382 11587 leader_election.cc:302] T 00000000000000000000000000000000 P 7ad50cbc15e44d428f073d146c382b4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ad50cbc15e44d428f073d146c382b4e; no voters:
I20260808 18:26:38.749691 11587 leader_election.cc:288] T 00000000000000000000000000000000 P 7ad50cbc15e44d428f073d146c382b4e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:26:38.749794 11592 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ad50cbc15e44d428f073d146c382b4e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:26:38.750062 11592 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ad50cbc15e44d428f073d146c382b4e [term 1 LEADER]: Becoming Leader. State: Replica: 7ad50cbc15e44d428f073d146c382b4e, State: Running, Role: LEADER
I20260808 18:26:38.750376 11592 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ad50cbc15e44d428f073d146c382b4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ad50cbc15e44d428f073d146c382b4e" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 33475 } }
I20260808 18:26:38.750867 11587 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ad50cbc15e44d428f073d146c382b4e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:26:38.752074 11593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ad50cbc15e44d428f073d146c382b4e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ad50cbc15e44d428f073d146c382b4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ad50cbc15e44d428f073d146c382b4e" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 33475 } } }
I20260808 18:26:38.752214 11593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ad50cbc15e44d428f073d146c382b4e [sys.catalog]: This master's current role is: LEADER
I20260808 18:26:38.752583 11601 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:26:38.752740 11594 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ad50cbc15e44d428f073d146c382b4e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ad50cbc15e44d428f073d146c382b4e. Latest consensus state: current_term: 1 leader_uuid: "7ad50cbc15e44d428f073d146c382b4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ad50cbc15e44d428f073d146c382b4e" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 33475 } } }
I20260808 18:26:38.752859 11594 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ad50cbc15e44d428f073d146c382b4e [sys.catalog]: This master's current role is: LEADER
I20260808 18:26:38.755771 11601 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:26:38.761214 11601 catalog_manager.cc:1347] Generated new cluster ID: 00a55f836d5c4755b47daefec6b4c5f1
I20260808 18:26:38.761287 11601 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:26:38.779016 11601 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:26:38.780175 11601 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:26:38.786546 11601 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ad50cbc15e44d428f073d146c382b4e: Generated new TSK 0
I20260808 18:26:38.787214 11601 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:26:38.791908 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.129:0
--local_ip_for_outbound_sockets=127.23.72.129
--webserver_interface=127.23.72.129
--webserver_port=0
--tserver_master_addrs=127.23.72.190:33475
--builtin_ntp_servers=127.23.72.148:33059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 18:26:38.911957 11613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:26:38.912236 11613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:26:38.912305 11613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:26:38.916416 11613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:26:38.916499 11613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:26:38.916594 11613 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.129
I20260808 18:26:38.921243 11613 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:33059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.72.129
--webserver_port=0
--tserver_master_addrs=127.23.72.190:33475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11613
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.129
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:26:38.922278 11613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:26:38.923733 11613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:26:38.933178 11619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:38.933157 11622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:38.933179 11620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:38.933445 11613 server_base.cc:1034] running on GCE node
I20260808 18:26:38.933876 11613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:26:38.934502 11613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:26:38.935683 11613 hybrid_clock.cc:648] HybridClock initialized: now 1786213598935664 us; error 30 us; skew 500 ppm
I20260808 18:26:38.937489 11613 webserver.cc:460] Webserver started at http://127.23.72.129:40825/ using document root <none> and password file <none>
I20260808 18:26:38.938069 11613 fs_manager.cc:359] Metadata directory not provided
I20260808 18:26:38.938162 11613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:26:38.938400 11613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:26:38.940565 11613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-0/data/instance:
uuid: "747f2bbac4b34ca999bdf0fa565b614d"
format_stamp: "Formatted at 2026-08-08 18:26:38 on dist-test-slave-5k54"
server_key: "172146630ebefcbb38ca52e461626fda"
server_key_iv: "92cd5cde2f02ac827ba49e19d97cc32a"
server_key_version: "encryptionkey@0"
I20260808 18:26:38.941128 11613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance:
uuid: "747f2bbac4b34ca999bdf0fa565b614d"
format_stamp: "Formatted at 2026-08-08 18:26:38 on dist-test-slave-5k54"
server_key: "172146630ebefcbb38ca52e461626fda"
server_key_iv: "92cd5cde2f02ac827ba49e19d97cc32a"
server_key_version: "encryptionkey@0"
I20260808 18:26:38.944975 11613 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 18:26:38.947329 11628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:38.948635 11613 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:26:38.948776 11613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-0/wal
uuid: "747f2bbac4b34ca999bdf0fa565b614d"
format_stamp: "Formatted at 2026-08-08 18:26:38 on dist-test-slave-5k54"
server_key: "172146630ebefcbb38ca52e461626fda"
server_key_iv: "92cd5cde2f02ac827ba49e19d97cc32a"
server_key_version: "encryptionkey@0"
I20260808 18:26:38.948889 11613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:26:38.972255 11613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:26:38.972959 11613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:26:38.973156 11613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:26:38.973750 11613 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:26:38.974753 11613 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:26:38.974824 11613 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:38.974897 11613 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:26:38.974943 11613 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:38.984139 11613 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.129:39611
I20260808 18:26:38.984170 11747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.129:39611 every 8 connection(s)
I20260808 18:26:38.985101 11613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
I20260808 18:26:38.993551 11590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:38.986839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:41119 (local address 127.23.72.190:33475)
0808 18:26:38.986989 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:38.986993 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:38.987989 (+ 996us) server_negotiation.cc:407] Connection header received
0808 18:26:38.989332 (+ 1343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:38.989334 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:38.989382 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:38.989455 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:38.990293 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:38.990812 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:38.991922 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:38.992137 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:38.992216 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:38.993166 (+ 950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:38.993169 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:38.993171 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:26:38.993182 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:38.993199 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:26:38.993207 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:38.993262 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:38.993394 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 18:26:38.993442 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 18:26:38.994330 11750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:38.987240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:33475 (local address 127.23.72.129:41119)
0808 18:26:38.987869 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:38.987908 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:38.989136 (+ 1228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:38.989612 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:38.989618 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:38.989699 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:26:38.990174 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:38.990186 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:38.990926 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:38.990928 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:38.991818 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:38.991826 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:38.992265 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:38.992267 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:38.992395 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:38.992936 (+ 541us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:26:38.992937 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:26:38.992975 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:26:38.993032 (+ 57us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:26:38.993279 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:26:38.993285 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:26:38.993289 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 18:26:38.993377 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 18:26:38.993580 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":133,"threads_started":1}
I20260808 18:26:38.994791 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 11613
I20260808 18:26:38.994943 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance
I20260808 18:26:38.995282 23842 external_mini_cluster.cc:1428] Setting key 3d0b6c492494d69112e078ce4b4845f0
I20260808 18:26:38.995684 11748 heartbeater.cc:347] Connected to a master server at 127.23.72.190:33475
I20260808 18:26:38.995937 11748 heartbeater.cc:464] Registering TS with master...
I20260808 18:26:38.996511 11748 heartbeater.cc:511] Master 127.23.72.190:33475 requested a full tablet report, sending...
I20260808 18:26:38.997393 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.130:0
--local_ip_for_outbound_sockets=127.23.72.130
--webserver_interface=127.23.72.130
--webserver_port=0
--tserver_master_addrs=127.23.72.190:33475
--builtin_ntp_servers=127.23.72.148:33059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 18:26:38.998051 11547 ts_manager.cc:194] Registered new tserver with Master: 747f2bbac4b34ca999bdf0fa565b614d (127.23.72.129:39611)
I20260808 18:26:38.999155 11547 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.72.129:41119
W20260808 18:26:39.108388 11751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:26:39.108645 11751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:26:39.108685 11751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:26:39.112661 11751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:26:39.112723 11751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:26:39.112778 11751 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.130
I20260808 18:26:39.117545 11751 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:33059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.72.130
--webserver_port=0
--tserver_master_addrs=127.23.72.190:33475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11751
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.130
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:26:39.118667 11751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:26:39.120095 11751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:26:39.129459 11757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:39.129524 11756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:39.129626 11760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:39.129776 11751 server_base.cc:1034] running on GCE node
I20260808 18:26:39.130129 11751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:26:39.130761 11751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:26:39.131961 11751 hybrid_clock.cc:648] HybridClock initialized: now 1786213599131935 us; error 43 us; skew 500 ppm
I20260808 18:26:39.133764 11751 webserver.cc:460] Webserver started at http://127.23.72.130:43975/ using document root <none> and password file <none>
I20260808 18:26:39.134351 11751 fs_manager.cc:359] Metadata directory not provided
I20260808 18:26:39.134454 11751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:26:39.134689 11751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:26:39.136744 11751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-1/data/instance:
uuid: "d5efd54d93f34a64a84b5dae1a5f1cfd"
format_stamp: "Formatted at 2026-08-08 18:26:39 on dist-test-slave-5k54"
server_key: "0124c2ead632dab7d8e4b7aaf4377251"
server_key_iv: "ce7e7cf05b270b5da34c6ea16ec45157"
server_key_version: "encryptionkey@0"
I20260808 18:26:39.137306 11751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance:
uuid: "d5efd54d93f34a64a84b5dae1a5f1cfd"
format_stamp: "Formatted at 2026-08-08 18:26:39 on dist-test-slave-5k54"
server_key: "0124c2ead632dab7d8e4b7aaf4377251"
server_key_iv: "ce7e7cf05b270b5da34c6ea16ec45157"
server_key_version: "encryptionkey@0"
I20260808 18:26:39.140882 11751 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:26:39.143186 11766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:39.144421 11751 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:26:39.144559 11751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-1/wal
uuid: "d5efd54d93f34a64a84b5dae1a5f1cfd"
format_stamp: "Formatted at 2026-08-08 18:26:39 on dist-test-slave-5k54"
server_key: "0124c2ead632dab7d8e4b7aaf4377251"
server_key_iv: "ce7e7cf05b270b5da34c6ea16ec45157"
server_key_version: "encryptionkey@0"
I20260808 18:26:39.144670 11751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:26:39.161263 11751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:26:39.161916 11751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:26:39.162114 11751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:26:39.162703 11751 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:26:39.163647 11751 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:26:39.163720 11751 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:39.163790 11751 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:26:39.163833 11751 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:39.172736 11751 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.130:33353
I20260808 18:26:39.172762 11883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.130:33353 every 8 connection(s)
I20260808 18:26:39.173635 11751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
I20260808 18:26:39.174539 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 11751
I20260808 18:26:39.174686 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance
I20260808 18:26:39.175004 23842 external_mini_cluster.cc:1428] Setting key 2b0ee8c0fc18f09df2ce9d80de1d587b
I20260808 18:26:39.177243 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.131:0
--local_ip_for_outbound_sockets=127.23.72.131
--webserver_interface=127.23.72.131
--webserver_port=0
--tserver_master_addrs=127.23.72.190:33475
--builtin_ntp_servers=127.23.72.148:33059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 18:26:39.184930 11590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:39.175594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:34147 (local address 127.23.72.190:33475)
0808 18:26:39.175737 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:39.175741 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:39.177064 (+ 1323us) server_negotiation.cc:407] Connection header received
0808 18:26:39.179140 (+ 2076us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:39.179146 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:39.179214 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:39.179320 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:39.180303 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:39.180922 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:39.182822 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:39.183038 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:39.183125 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:39.184458 (+ 1333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:39.184461 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:39.184463 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:26:39.184475 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:39.184496 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:26:39.184504 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:39.184564 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:39.184769 (+ 205us) server_negotiation.cc:299] Negotiation successful
0808 18:26:39.184820 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 18:26:39.186192 11886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:39.176082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:33475 (local address 127.23.72.130:34147)
0808 18:26:39.176874 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:39.176945 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:39.178879 (+ 1934us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:39.179560 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:39.179567 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:39.179649 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:26:39.180148 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:39.180164 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:39.181063 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:39.181069 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:39.182690 (+ 1621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:39.182704 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:39.183205 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:39.183211 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:39.183502 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:39.184143 (+ 641us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:26:39.184148 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:26:39.184208 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:26:39.184297 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:26:39.184593 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:26:39.184605 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:26:39.184612 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 18:26:39.184745 (+ 133us) client_negotiation.cc:240] Negotiation successful
0808 18:26:39.185079 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":149,"threads_started":1}
I20260808 18:26:39.187350 11884 heartbeater.cc:347] Connected to a master server at 127.23.72.190:33475
I20260808 18:26:39.187597 11884 heartbeater.cc:464] Registering TS with master...
I20260808 18:26:39.188087 11884 heartbeater.cc:511] Master 127.23.72.190:33475 requested a full tablet report, sending...
I20260808 18:26:39.189121 11547 ts_manager.cc:194] Registered new tserver with Master: d5efd54d93f34a64a84b5dae1a5f1cfd (127.23.72.130:33353)
I20260808 18:26:39.190004 11547 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.72.130:34147
W20260808 18:26:39.291788 11888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:26:39.292030 11888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:26:39.292068 11888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:26:39.295956 11888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:26:39.296002 11888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:26:39.296053 11888 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.131
I20260808 18:26:39.300609 11888 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:33059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.72.131
--webserver_port=0
--tserver_master_addrs=127.23.72.190:33475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11888
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.131
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:26:39.301540 11888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:26:39.302855 11888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:26:39.312129 11893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:39.312093 11894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:39.312074 11888 server_base.cc:1034] running on GCE node
W20260808 18:26:39.312074 11896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:39.312690 11888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:26:39.313391 11888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:26:39.314558 11888 hybrid_clock.cc:648] HybridClock initialized: now 1786213599314538 us; error 29 us; skew 500 ppm
I20260808 18:26:39.316463 11888 webserver.cc:460] Webserver started at http://127.23.72.131:43161/ using document root <none> and password file <none>
I20260808 18:26:39.317047 11888 fs_manager.cc:359] Metadata directory not provided
I20260808 18:26:39.317109 11888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:26:39.317348 11888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:26:39.319337 11888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-2/data/instance:
uuid: "66fed47fdab6423cab1ae16a122edba5"
format_stamp: "Formatted at 2026-08-08 18:26:39 on dist-test-slave-5k54"
server_key: "b1a9720a431d4fc4d591ed82f506496d"
server_key_iv: "dcbf1bee4a2b84a7ff4404d4b661c611"
server_key_version: "encryptionkey@0"
I20260808 18:26:39.320020 11888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance:
uuid: "66fed47fdab6423cab1ae16a122edba5"
format_stamp: "Formatted at 2026-08-08 18:26:39 on dist-test-slave-5k54"
server_key: "b1a9720a431d4fc4d591ed82f506496d"
server_key_iv: "dcbf1bee4a2b84a7ff4404d4b661c611"
server_key_version: "encryptionkey@0"
I20260808 18:26:39.323812 11888 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 18:26:39.326370 11902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:39.327675 11888 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:26:39.327795 11888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-2/wal
uuid: "66fed47fdab6423cab1ae16a122edba5"
format_stamp: "Formatted at 2026-08-08 18:26:39 on dist-test-slave-5k54"
server_key: "b1a9720a431d4fc4d591ed82f506496d"
server_key_iv: "dcbf1bee4a2b84a7ff4404d4b661c611"
server_key_version: "encryptionkey@0"
I20260808 18:26:39.327921 11888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:26:39.337596 11888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:26:39.338284 11888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:26:39.338486 11888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:26:39.339078 11888 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:26:39.340093 11888 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:26:39.340139 11888 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:39.340206 11888 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:26:39.340240 11888 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:39.349071 12023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.131:44613 every 8 connection(s)
I20260808 18:26:39.349077 11888 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.131:44613
I20260808 18:26:39.350035 11888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
I20260808 18:26:39.354616 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 11888
I20260808 18:26:39.354745 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance
I20260808 18:26:39.355028 23842 external_mini_cluster.cc:1428] Setting key 9b835820693765eeffbbc7a8df2c6347
I20260808 18:26:39.359951 11590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:39.351921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:58471 (local address 127.23.72.190:33475)
0808 18:26:39.352088 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:39.352091 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:39.353245 (+ 1154us) server_negotiation.cc:407] Connection header received
0808 18:26:39.354736 (+ 1491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:39.354739 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:39.354797 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:39.354949 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:39.356096 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:39.356715 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:39.357949 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:39.358179 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:39.358291 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:39.359450 (+ 1159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:39.359454 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:39.359457 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:26:39.359470 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:39.359491 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:26:39.359507 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:39.359585 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:39.359746 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 18:26:39.359803 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 18:26:39.360728 12026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:39.352398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:33475 (local address 127.23.72.131:58471)
0808 18:26:39.353098 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:39.353138 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:39.354524 (+ 1386us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:39.355146 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:39.355156 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:39.355252 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:26:39.355855 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:39.355869 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:39.356838 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:39.356842 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:39.357813 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:39.357823 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:39.358326 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:39.358329 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:39.358507 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:39.359140 (+ 633us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:26:39.359141 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:26:39.359179 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:26:39.359238 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:26:39.359587 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:26:39.359599 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:26:39.359605 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 18:26:39.359733 (+ 128us) client_negotiation.cc:240] Negotiation successful
0808 18:26:39.360000 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":111,"threads_started":1}
I20260808 18:26:39.361742 12024 heartbeater.cc:347] Connected to a master server at 127.23.72.190:33475
I20260808 18:26:39.361984 12024 heartbeater.cc:464] Registering TS with master...
I20260808 18:26:39.362486 12024 heartbeater.cc:511] Master 127.23.72.190:33475 requested a full tablet report, sending...
I20260808 18:26:39.363481 11547 ts_manager.cc:194] Registered new tserver with Master: 66fed47fdab6423cab1ae16a122edba5 (127.23.72.131:44613)
I20260808 18:26:39.364060 11547 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.72.131:58471
I20260808 18:26:39.369024 23842 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:26:39.374761 11590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:39.371142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46720 (local address 127.23.72.190:33475)
0808 18:26:39.371269 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:39.371273 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:39.371293 (+ 20us) server_negotiation.cc:407] Connection header received
0808 18:26:39.371385 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:39.371387 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:39.371461 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:39.371546 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:39.371802 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:39.372400 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:39.373241 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:39.373442 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:39.373514 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:39.374370 (+ 856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:39.374373 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:39.374375 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:26:39.374385 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:39.374401 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:26:39.374408 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:39.374498 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:39.374628 (+ 130us) server_negotiation.cc:299] Negotiation successful
0808 18:26:39.374659 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 18:26:39.377453 11547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46720:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:26:39.380565 11547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:26:39.395232 12038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:39.389131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.130:33353 (local address 127.0.0.1:51048)
0808 18:26:39.389997 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:39.390051 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:39.390193 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:39.391013 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:39.391018 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:39.391051 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:26:39.391381 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:39.391388 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:39.392846 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:39.392849 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:39.393840 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:39.393848 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:39.394646 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:39.394649 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:39.394812 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:39.394838 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 18:26:39.394906 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 18:26:39.395035 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"mutex_wait_us":262,"spinlock_wait_cycles":18048,"thread_start_us":100,"threads_started":1}
I20260808 18:26:39.395267 12036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:39.389252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51048 (local address 127.23.72.130:33353)
0808 18:26:39.390495 (+ 1243us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:39.390508 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:39.390534 (+ 26us) server_negotiation.cc:407] Connection header received
0808 18:26:39.390610 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:39.390615 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:39.390815 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:39.391198 (+ 383us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:26:39.391535 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:39.392719 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:39.393973 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:39.394495 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:39.394763 (+ 268us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:39.394968 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:39.395021 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 18:26:39.395117 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1129,"thread_start_us":95,"threads_started":1}
I20260808 18:26:39.395233 12037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:39.389245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.131:44613 (local address 127.0.0.1:46896)
0808 18:26:39.389998 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:39.390053 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:39.390196 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:39.391015 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:39.391021 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:39.391054 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:26:39.391213 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:39.391220 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:39.392324 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:39.392327 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:39.393506 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:39.393516 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:39.394579 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:39.394585 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:39.394715 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:39.394777 (+ 62us) client_negotiation.cc:769] Sending connection context
0808 18:26:39.394883 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 18:26:39.395040 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"mutex_wait_us":40,"spinlock_wait_cycles":18688,"thread_start_us":108,"threads_started":1}
I20260808 18:26:39.395953 12035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:39.389528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:39611 (local address 127.0.0.1:52368)
0808 18:26:39.390730 (+ 1202us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:39.390745 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:39.390833 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:39.391354 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:39.391357 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:39.391374 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:26:39.391478 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:39.391485 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:39.393048 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:39.393060 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:39.394230 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:39.394244 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:39.395530 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:39.395534 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:39.395641 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:39.395667 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 18:26:39.395748 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 18:26:39.395822 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1141,"mutex_wait_us":1,"thread_start_us":96,"threads_started":1}
I20260808 18:26:39.396076 12039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:39.389953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46896 (local address 127.23.72.131:44613)
0808 18:26:39.390637 (+ 684us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:39.390647 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:39.390665 (+ 18us) server_negotiation.cc:407] Connection header received
0808 18:26:39.390726 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:39.390730 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:39.390875 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:39.391027 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:26:39.391349 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:39.392199 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:39.393637 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:39.394450 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:39.394554 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:39.394740 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:39.395278 (+ 538us) server_negotiation.cc:299] Negotiation successful
0808 18:26:39.395909 (+ 631us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":595,"thread_start_us":164,"threads_started":1}
I20260808 18:26:39.396440 12040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:39.390084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52368 (local address 127.23.72.129:39611)
0808 18:26:39.390583 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:39.390597 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:39.390823 (+ 226us) server_negotiation.cc:407] Connection header received
0808 18:26:39.390971 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:39.390979 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:39.391186 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:39.391359 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:26:39.391576 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:39.392719 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:39.394383 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:39.395201 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:39.395306 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:39.395539 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:39.396184 (+ 645us) server_negotiation.cc:299] Negotiation successful
0808 18:26:39.396287 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":124,"threads_started":1}
I20260808 18:26:39.398602 11815 tablet_service.cc:1467] Processing CreateTablet for tablet 57da7873868c444fa868d34277759bbe (DEFAULT_TABLE table=test-table [id=a0257f83d78547459494654e15c61bba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:26:39.398824 11680 tablet_service.cc:1467] Processing CreateTablet for tablet 57da7873868c444fa868d34277759bbe (DEFAULT_TABLE table=test-table [id=a0257f83d78547459494654e15c61bba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:26:39.398823 11955 tablet_service.cc:1467] Processing CreateTablet for tablet 57da7873868c444fa868d34277759bbe (DEFAULT_TABLE table=test-table [id=a0257f83d78547459494654e15c61bba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:26:39.399719 11815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57da7873868c444fa868d34277759bbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:26:39.399853 11680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57da7873868c444fa868d34277759bbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:26:39.399853 11955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57da7873868c444fa868d34277759bbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:26:39.407186 12042 tablet_bootstrap.cc:492] T 57da7873868c444fa868d34277759bbe P 66fed47fdab6423cab1ae16a122edba5: Bootstrap starting.
I20260808 18:26:39.407648 12041 tablet_bootstrap.cc:492] T 57da7873868c444fa868d34277759bbe P 747f2bbac4b34ca999bdf0fa565b614d: Bootstrap starting.
I20260808 18:26:39.408474 12043 tablet_bootstrap.cc:492] T 57da7873868c444fa868d34277759bbe P d5efd54d93f34a64a84b5dae1a5f1cfd: Bootstrap starting.
I20260808 18:26:39.409880 12042 tablet_bootstrap.cc:654] T 57da7873868c444fa868d34277759bbe P 66fed47fdab6423cab1ae16a122edba5: Neither blocks nor log segments found. Creating new log.
I20260808 18:26:39.410004 12041 tablet_bootstrap.cc:654] T 57da7873868c444fa868d34277759bbe P 747f2bbac4b34ca999bdf0fa565b614d: Neither blocks nor log segments found. Creating new log.
I20260808 18:26:39.410457 12043 tablet_bootstrap.cc:654] T 57da7873868c444fa868d34277759bbe P d5efd54d93f34a64a84b5dae1a5f1cfd: Neither blocks nor log segments found. Creating new log.
I20260808 18:26:39.410734 12042 log.cc:824] T 57da7873868c444fa868d34277759bbe P 66fed47fdab6423cab1ae16a122edba5: Log is configured to *not* fsync() on all Append() calls
I20260808 18:26:39.410862 12041 log.cc:824] T 57da7873868c444fa868d34277759bbe P 747f2bbac4b34ca999bdf0fa565b614d: Log is configured to *not* fsync() on all Append() calls
I20260808 18:26:39.411217 12043 log.cc:824] T 57da7873868c444fa868d34277759bbe P d5efd54d93f34a64a84b5dae1a5f1cfd: Log is configured to *not* fsync() on all Append() calls
I20260808 18:26:39.412923 12042 tablet_bootstrap.cc:492] T 57da7873868c444fa868d34277759bbe P 66fed47fdab6423cab1ae16a122edba5: No bootstrap required, opened a new log
I20260808 18:26:39.412937 12043 tablet_bootstrap.cc:492] T 57da7873868c444fa868d34277759bbe P d5efd54d93f34a64a84b5dae1a5f1cfd: No bootstrap required, opened a new log
I20260808 18:26:39.413122 12043 ts_tablet_manager.cc:1397] T 57da7873868c444fa868d34277759bbe P d5efd54d93f34a64a84b5dae1a5f1cfd: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 18:26:39.413122 12042 ts_tablet_manager.cc:1397] T 57da7873868c444fa868d34277759bbe P 66fed47fdab6423cab1ae16a122edba5: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260808 18:26:39.413421 12041 tablet_bootstrap.cc:492] T 57da7873868c444fa868d34277759bbe P 747f2bbac4b34ca999bdf0fa565b614d: No bootstrap required, opened a new log
I20260808 18:26:39.413622 12041 ts_tablet_manager.cc:1397] T 57da7873868c444fa868d34277759bbe P 747f2bbac4b34ca999bdf0fa565b614d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 18:26:39.416538 12041 raft_consensus.cc:348] T 57da7873868c444fa868d34277759bbe P 747f2bbac4b34ca999bdf0fa565b614d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5efd54d93f34a64a84b5dae1a5f1cfd" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 33353 } } peers { permanent_uuid: "66fed47fdab6423cab1ae16a122edba5" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 44613 } } peers { permanent_uuid: "747f2bbac4b34ca999bdf0fa565b614d" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 39611 } }
I20260808 18:26:39.416777 12041 raft_consensus.cc:374] T 57da7873868c444fa868d34277759bbe P 747f2bbac4b34ca999bdf0fa565b614d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:26:39.416831 12041 raft_consensus.cc:729] T 57da7873868c444fa868d34277759bbe P 747f2bbac4b34ca999bdf0fa565b614d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 747f2bbac4b34ca999bdf0fa565b614d, State: Initialized, Role: FOLLOWER
I20260808 18:26:39.417266 12041 consensus_queue.cc:260] T 57da7873868c444fa868d34277759bbe P 747f2bbac4b34ca999bdf0fa565b614d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5efd54d93f34a64a84b5dae1a5f1cfd" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 33353 } } peers { permanent_uuid: "66fed47fdab6423cab1ae16a122edba5" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 44613 } } peers { permanent_uuid: "747f2bbac4b34ca999bdf0fa565b614d" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 39611 } }
I20260808 18:26:39.417443 12043 raft_consensus.cc:348] T 57da7873868c444fa868d34277759bbe P d5efd54d93f34a64a84b5dae1a5f1cfd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5efd54d93f34a64a84b5dae1a5f1cfd" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 33353 } } peers { permanent_uuid: "66fed47fdab6423cab1ae16a122edba5" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 44613 } } peers { permanent_uuid: "747f2bbac4b34ca999bdf0fa565b614d" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 39611 } }
I20260808 18:26:39.417443 12042 raft_consensus.cc:348] T 57da7873868c444fa868d34277759bbe P 66fed47fdab6423cab1ae16a122edba5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5efd54d93f34a64a84b5dae1a5f1cfd" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 33353 } } peers { permanent_uuid: "66fed47fdab6423cab1ae16a122edba5" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 44613 } } peers { permanent_uuid: "747f2bbac4b34ca999bdf0fa565b614d" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 39611 } }
I20260808 18:26:39.417695 12042 raft_consensus.cc:374] T 57da7873868c444fa868d34277759bbe P 66fed47fdab6423cab1ae16a122edba5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:26:39.417695 12043 raft_consensus.cc:374] T 57da7873868c444fa868d34277759bbe P d5efd54d93f34a64a84b5dae1a5f1cfd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:26:39.417757 12043 raft_consensus.cc:729] T 57da7873868c444fa868d34277759bbe P d5efd54d93f34a64a84b5dae1a5f1cfd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5efd54d93f34a64a84b5dae1a5f1cfd, State: Initialized, Role: FOLLOWER
I20260808 18:26:39.417764 12042 raft_consensus.cc:729] T 57da7873868c444fa868d34277759bbe P 66fed47fdab6423cab1ae16a122edba5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66fed47fdab6423cab1ae16a122edba5, State: Initialized, Role: FOLLOWER
I20260808 18:26:39.418102 11748 heartbeater.cc:503] Master 127.23.72.190:33475 was elected leader, sending a full tablet report...
I20260808 18:26:39.418227 12043 consensus_queue.cc:260] T 57da7873868c444fa868d34277759bbe P d5efd54d93f34a64a84b5dae1a5f1cfd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5efd54d93f34a64a84b5dae1a5f1cfd" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 33353 } } peers { permanent_uuid: "66fed47fdab6423cab1ae16a122edba5" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 44613 } } peers { permanent_uuid: "747f2bbac4b34ca999bdf0fa565b614d" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 39611 } }
I20260808 18:26:39.418227 12042 consensus_queue.cc:260] T 57da7873868c444fa868d34277759bbe P 66fed47fdab6423cab1ae16a122edba5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5efd54d93f34a64a84b5dae1a5f1cfd" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 33353 } } peers { permanent_uuid: "66fed47fdab6423cab1ae16a122edba5" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 44613 } } peers { permanent_uuid: "747f2bbac4b34ca999bdf0fa565b614d" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 39611 } }
I20260808 18:26:39.419209 12042 ts_tablet_manager.cc:1428] T 57da7873868c444fa868d34277759bbe P 66fed47fdab6423cab1ae16a122edba5: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 18:26:39.419209 12043 ts_tablet_manager.cc:1428] T 57da7873868c444fa868d34277759bbe P d5efd54d93f34a64a84b5dae1a5f1cfd: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 18:26:39.419214 12041 ts_tablet_manager.cc:1428] T 57da7873868c444fa868d34277759bbe P 747f2bbac4b34ca999bdf0fa565b614d: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 18:26:39.419554 12024 heartbeater.cc:503] Master 127.23.72.190:33475 was elected leader, sending a full tablet report...
I20260808 18:26:39.419554 11884 heartbeater.cc:503] Master 127.23.72.190:33475 was elected leader, sending a full tablet report...
W20260808 18:26:39.424526 11885 tablet.cc:2369] T 57da7873868c444fa868d34277759bbe P d5efd54d93f34a64a84b5dae1a5f1cfd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:26:39.448621 12050 raft_consensus.cc:482] T 57da7873868c444fa868d34277759bbe P 66fed47fdab6423cab1ae16a122edba5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:26:39.448881 12050 raft_consensus.cc:504] T 57da7873868c444fa868d34277759bbe P 66fed47fdab6423cab1ae16a122edba5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5efd54d93f34a64a84b5dae1a5f1cfd" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 33353 } } peers { permanent_uuid: "66fed47fdab6423cab1ae16a122edba5" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 44613 } } peers { permanent_uuid: "747f2bbac4b34ca999bdf0fa565b614d" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 39611 } }
I20260808 18:26:39.450233 12050 leader_election.cc:288] T 57da7873868c444fa868d34277759bbe P 66fed47fdab6423cab1ae16a122edba5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5efd54d93f34a64a84b5dae1a5f1cfd (127.23.72.130:33353), 747f2bbac4b34ca999bdf0fa565b614d (127.23.72.129:39611)
I20260808 18:26:39.454221 12026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:39.450453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:39611 (local address 127.23.72.131:35833)
0808 18:26:39.450594 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:39.450612 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:39.450705 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:39.451015 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:39.451018 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:39.451060 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:26:39.451216 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:39.451224 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:39.452241 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:39.452248 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:39.453134 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:39.453143 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:39.453786 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:39.453790 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:39.453905 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:39.453924 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 18:26:39.454012 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 18:26:39.454090 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"spinlock_wait_cycles":384}
I20260808 18:26:39.454221 12040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:39.450556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:35833 (local address 127.23.72.129:39611)
0808 18:26:39.450685 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:39.450690 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:39.450708 (+ 18us) server_negotiation.cc:407] Connection header received
0808 18:26:39.450852 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:39.450855 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:39.450909 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:39.451001 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:26:39.451369 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:39.452122 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:39.453257 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:39.453659 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:39.453805 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:39.453859 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:39.454029 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 18:26:39.454096 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 18:26:39.454602 12036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:39.450549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:54689 (local address 127.23.72.130:33353)
0808 18:26:39.450686 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:39.450691 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:39.451069 (+ 378us) server_negotiation.cc:407] Connection header received
0808 18:26:39.451204 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:39.451207 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:39.451260 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:39.451340 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:26:39.451641 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:39.452255 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:39.453539 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:39.454034 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:39.454163 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:39.454208 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:39.454420 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 18:26:39.454487 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 18:26:39.454974 12051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:39.450452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.130:33353 (local address 127.23.72.131:54689)
0808 18:26:39.450960 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:39.450977 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:39.451084 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:39.451367 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:39.451370 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:39.451388 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:26:39.451523 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:39.451530 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:39.452362 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:39.452367 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:39.453421 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:39.453432 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:39.454163 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:39.454167 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:39.454265 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:39.454289 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 18:26:39.454736 (+ 447us) client_negotiation.cc:240] Negotiation successful
0808 18:26:39.454815 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"mutex_wait_us":152,"spinlock_wait_cycles":5504,"thread_start_us":134,"threads_started":1}
I20260808 18:26:39.454909 11700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57da7873868c444fa868d34277759bbe" candidate_uuid: "66fed47fdab6423cab1ae16a122edba5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "747f2bbac4b34ca999bdf0fa565b614d" is_pre_election: true
I20260808 18:26:39.455242 11700 raft_consensus.cc:2454] T 57da7873868c444fa868d34277759bbe P 747f2bbac4b34ca999bdf0fa565b614d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66fed47fdab6423cab1ae16a122edba5 in term 0.
I20260808 18:26:39.455639 11836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57da7873868c444fa868d34277759bbe" candidate_uuid: "66fed47fdab6423cab1ae16a122edba5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5efd54d93f34a64a84b5dae1a5f1cfd" is_pre_election: true
I20260808 18:26:39.455825 11906 leader_election.cc:302] T 57da7873868c444fa868d34277759bbe P 66fed47fdab6423cab1ae16a122edba5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66fed47fdab6423cab1ae16a122edba5, 747f2bbac4b34ca999bdf0fa565b614d; no voters:
I20260808 18:26:39.455966 11836 raft_consensus.cc:2454] T 57da7873868c444fa868d34277759bbe P d5efd54d93f34a64a84b5dae1a5f1cfd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66fed47fdab6423cab1ae16a122edba5 in term 0.
I20260808 18:26:39.456141 12050 raft_consensus.cc:2781] T 57da7873868c444fa868d34277759bbe P 66fed47fdab6423cab1ae16a122edba5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:26:39.456243 12050 raft_consensus.cc:482] T 57da7873868c444fa868d34277759bbe P 66fed47fdab6423cab1ae16a122edba5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:26:39.456310 12050 raft_consensus.cc:3037] T 57da7873868c444fa868d34277759bbe P 66fed47fdab6423cab1ae16a122edba5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:26:39.457340 12050 raft_consensus.cc:504] T 57da7873868c444fa868d34277759bbe P 66fed47fdab6423cab1ae16a122edba5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5efd54d93f34a64a84b5dae1a5f1cfd" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 33353 } } peers { permanent_uuid: "66fed47fdab6423cab1ae16a122edba5" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 44613 } } peers { permanent_uuid: "747f2bbac4b34ca999bdf0fa565b614d" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 39611 } }
I20260808 18:26:39.457706 12050 leader_election.cc:288] T 57da7873868c444fa868d34277759bbe P 66fed47fdab6423cab1ae16a122edba5 [CANDIDATE]: Term 1 election: Requested vote from peers d5efd54d93f34a64a84b5dae1a5f1cfd (127.23.72.130:33353), 747f2bbac4b34ca999bdf0fa565b614d (127.23.72.129:39611)
I20260808 18:26:39.458035 11700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57da7873868c444fa868d34277759bbe" candidate_uuid: "66fed47fdab6423cab1ae16a122edba5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "747f2bbac4b34ca999bdf0fa565b614d"
I20260808 18:26:39.458148 11700 raft_consensus.cc:3037] T 57da7873868c444fa868d34277759bbe P 747f2bbac4b34ca999bdf0fa565b614d [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:26:39.458249 11836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57da7873868c444fa868d34277759bbe" candidate_uuid: "66fed47fdab6423cab1ae16a122edba5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5efd54d93f34a64a84b5dae1a5f1cfd"
I20260808 18:26:39.458439 11836 raft_consensus.cc:3037] T 57da7873868c444fa868d34277759bbe P d5efd54d93f34a64a84b5dae1a5f1cfd [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:26:39.459317 11700 raft_consensus.cc:2454] T 57da7873868c444fa868d34277759bbe P 747f2bbac4b34ca999bdf0fa565b614d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66fed47fdab6423cab1ae16a122edba5 in term 1.
I20260808 18:26:39.459558 11836 raft_consensus.cc:2454] T 57da7873868c444fa868d34277759bbe P d5efd54d93f34a64a84b5dae1a5f1cfd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66fed47fdab6423cab1ae16a122edba5 in term 1.
I20260808 18:26:39.459676 11906 leader_election.cc:302] T 57da7873868c444fa868d34277759bbe P 66fed47fdab6423cab1ae16a122edba5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66fed47fdab6423cab1ae16a122edba5, 747f2bbac4b34ca999bdf0fa565b614d; no voters:
I20260808 18:26:39.459863 12050 raft_consensus.cc:2781] T 57da7873868c444fa868d34277759bbe P 66fed47fdab6423cab1ae16a122edba5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:26:39.460094 12050 raft_consensus.cc:686] T 57da7873868c444fa868d34277759bbe P 66fed47fdab6423cab1ae16a122edba5 [term 1 LEADER]: Becoming Leader. State: Replica: 66fed47fdab6423cab1ae16a122edba5, State: Running, Role: LEADER
I20260808 18:26:39.460443 12050 consensus_queue.cc:237] T 57da7873868c444fa868d34277759bbe P 66fed47fdab6423cab1ae16a122edba5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5efd54d93f34a64a84b5dae1a5f1cfd" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 33353 } } peers { permanent_uuid: "66fed47fdab6423cab1ae16a122edba5" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 44613 } } peers { permanent_uuid: "747f2bbac4b34ca999bdf0fa565b614d" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 39611 } }
I20260808 18:26:39.463608 11547 catalog_manager.cc:5495] T 57da7873868c444fa868d34277759bbe P 66fed47fdab6423cab1ae16a122edba5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 66fed47fdab6423cab1ae16a122edba5 (127.23.72.131). New cstate: current_term: 1 leader_uuid: "66fed47fdab6423cab1ae16a122edba5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5efd54d93f34a64a84b5dae1a5f1cfd" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 33353 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66fed47fdab6423cab1ae16a122edba5" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 44613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "747f2bbac4b34ca999bdf0fa565b614d" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 39611 } health_report { overall_health: UNKNOWN } } }
W20260808 18:26:39.486204 11749 tablet.cc:2369] T 57da7873868c444fa868d34277759bbe P 747f2bbac4b34ca999bdf0fa565b614d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:26:39.493603 12039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:39.489945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46902 (local address 127.23.72.131:44613)
0808 18:26:39.490095 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:39.490099 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:39.490114 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:26:39.490285 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:39.490288 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:39.490357 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:39.490467 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:26:39.490740 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:39.491237 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:39.492200 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:39.492420 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:39.492529 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:39.492818 (+ 289us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:26:39.493201 (+ 383us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:26:39.493367 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:39.493430 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 18:26:39.493462 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 18:26:39.499327 11700 raft_consensus.cc:1261] T 57da7873868c444fa868d34277759bbe P 747f2bbac4b34ca999bdf0fa565b614d [term 1 FOLLOWER]: Refusing update from remote peer 66fed47fdab6423cab1ae16a122edba5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:26:39.499327 11836 raft_consensus.cc:1261] T 57da7873868c444fa868d34277759bbe P d5efd54d93f34a64a84b5dae1a5f1cfd [term 1 FOLLOWER]: Refusing update from remote peer 66fed47fdab6423cab1ae16a122edba5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:26:39.500023 12050 consensus_queue.cc:1035] T 57da7873868c444fa868d34277759bbe P 66fed47fdab6423cab1ae16a122edba5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "747f2bbac4b34ca999bdf0fa565b614d" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 39611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:26:39.500239 12052 consensus_queue.cc:1035] T 57da7873868c444fa868d34277759bbe P 66fed47fdab6423cab1ae16a122edba5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5efd54d93f34a64a84b5dae1a5f1cfd" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 33353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:26:39.511521 12057 mvcc.cc:205] Tried to move back new op lower bound from 7316330903542611968 to 7316330903392583680. Current Snapshot: MvccSnapshot[applied={T|T < 7316330903542611968}]
I20260808 18:26:39.515722 11547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:26:39.515869 11547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:26:39.518234 11547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7ad50cbc15e44d428f073d146c382b4e: Sending DeleteTablet for 3 replicas of tablet 57da7873868c444fa868d34277759bbe
I20260808 18:26:39.519102 11815 tablet_service.cc:1514] Processing DeleteTablet for tablet 57da7873868c444fa868d34277759bbe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:26:39 UTC) from {username='slave'} at 127.0.0.1:51048
I20260808 18:26:39.519114 11680 tablet_service.cc:1514] Processing DeleteTablet for tablet 57da7873868c444fa868d34277759bbe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:26:39 UTC) from {username='slave'} at 127.0.0.1:52368
I20260808 18:26:39.519696 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 11613
I20260808 18:26:39.519737 11955 tablet_service.cc:1514] Processing DeleteTablet for tablet 57da7873868c444fa868d34277759bbe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:26:39 UTC) from {username='slave'} at 127.0.0.1:46896
I20260808 18:26:39.525936 12068 tablet_replica.cc:330] T 57da7873868c444fa868d34277759bbe P 66fed47fdab6423cab1ae16a122edba5: stopping tablet replica
I20260808 18:26:39.526274 12068 raft_consensus.cc:2229] T 57da7873868c444fa868d34277759bbe P 66fed47fdab6423cab1ae16a122edba5 [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:26:39.526433 12066 tablet_replica.cc:330] T 57da7873868c444fa868d34277759bbe P d5efd54d93f34a64a84b5dae1a5f1cfd: stopping tablet replica
I20260808 18:26:39.526700 12066 raft_consensus.cc:2229] T 57da7873868c444fa868d34277759bbe P d5efd54d93f34a64a84b5dae1a5f1cfd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:26:39.526710 12068 raft_consensus.cc:2258] T 57da7873868c444fa868d34277759bbe P 66fed47fdab6423cab1ae16a122edba5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:26:39.527019 12066 raft_consensus.cc:2258] T 57da7873868c444fa868d34277759bbe P d5efd54d93f34a64a84b5dae1a5f1cfd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:26:39.528313 12068 ts_tablet_manager.cc:1905] T 57da7873868c444fa868d34277759bbe P 66fed47fdab6423cab1ae16a122edba5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:26:39.531850 12068 ts_tablet_manager.cc:1918] T 57da7873868c444fa868d34277759bbe P 66fed47fdab6423cab1ae16a122edba5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:26:39.531957 12068 log.cc:1196] T 57da7873868c444fa868d34277759bbe P 66fed47fdab6423cab1ae16a122edba5: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213345431840-23842-0/minicluster-data/ts-2/wal/wals/57da7873868c444fa868d34277759bbe
I20260808 18:26:39.532279 12068 ts_tablet_manager.cc:1939] T 57da7873868c444fa868d34277759bbe P 66fed47fdab6423cab1ae16a122edba5: Deleting consensus metadata
I20260808 18:26:39.532753 12066 ts_tablet_manager.cc:1905] T 57da7873868c444fa868d34277759bbe P d5efd54d93f34a64a84b5dae1a5f1cfd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:26:39.533514 11531 catalog_manager.cc:4841] TS 66fed47fdab6423cab1ae16a122edba5 (127.23.72.131:44613): tablet 57da7873868c444fa868d34277759bbe (table test-table [id=a0257f83d78547459494654e15c61bba]) successfully deleted
W20260808 18:26:39.533914 11534 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.72.129:39611 (error 108)
I20260808 18:26:39.534205 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 11751
I20260808 18:26:39.540031 12035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:39.539635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:39611 (local address 127.0.0.1:52384)
0808 18:26:39.539844 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:39.539926 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.129:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
W20260808 18:26:39.540215 11534 catalog_manager.cc:4584] TS 747f2bbac4b34ca999bdf0fa565b614d (127.23.72.129:39611): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 57da7873868c444fa868d34277759bbe: Network error: Client connection negotiation failed: client connection to 127.23.72.129:39611: connect: Connection refused (error 111)
I20260808 18:26:39.544091 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 11888
I20260808 18:26:39.546494 12035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:39.544220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.130:33353 (local address 127.0.0.1:51064)
0808 18:26:39.546317 (+ 2097us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:39.546388 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.130:33353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1989}
I20260808 18:26:39.552338 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 11513
2026-08-08T18:26:39Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1033 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:26:39 dist-test-slave-5k54 krb5kdc[12072](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:26:39 dist-test-slave-5k54 krb5kdc[12072](info): set up 2 sockets
Aug 08 18:26:39 dist-test-slave-5k54 krb5kdc[12072](info): commencing operation
krb5kdc: starting...
W20260808 18:26:40.636272 23842 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Aug 08 18:26:40 dist-test-slave-5k54 krb5kdc[12072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:26:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:26:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.72.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
I20260808 18:26:40.766036 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:35339
--webserver_interface=127.23.72.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:40433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:35339
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:26:40.889262 12085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:26:40.889576 12085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:26:40.889675 12085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:26:40.894017 12085 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:26:40.894104 12085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:26:40.894143 12085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:26:40.894200 12085 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:26:40.894237 12085 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:26:40.899405 12085 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:40433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:35339
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:35339
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.72.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12085
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:26:40.900470 12085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:26:40.901466 12085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:26:40.909544 12090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:40.909575 12091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:40.909561 12093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:40.909907 12085 server_base.cc:1034] running on GCE node
I20260808 18:26:40.910470 12085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:26:40.911564 12085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:26:40.912753 12085 hybrid_clock.cc:648] HybridClock initialized: now 1786213600912733 us; error 36 us; skew 500 ppm
Aug 08 18:26:40 dist-test-slave-5k54 krb5kdc[12072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213600, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:26:40.915489 12085 init.cc:377] Logged in from keytab as kudu/127.23.72.190@KRBTEST.COM (short username kudu)
I20260808 18:26:40.916355 12085 webserver.cc:460] Webserver started at http://127.23.72.190:40005/ using document root <none> and password file <none>
I20260808 18:26:40.916898 12085 fs_manager.cc:359] Metadata directory not provided
I20260808 18:26:40.916957 12085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:26:40.917191 12085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:26:40.919190 12085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "5ea358857ebb435b951751d828ba27c9"
format_stamp: "Formatted at 2026-08-08 18:26:40 on dist-test-slave-5k54"
server_key: "c453813c9f39351f71f40728cd1b8de7"
server_key_iv: "6862d722e742a45e1193268e5b1f089f"
server_key_version: "encryptionkey@0"
I20260808 18:26:40.919809 12085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "5ea358857ebb435b951751d828ba27c9"
format_stamp: "Formatted at 2026-08-08 18:26:40 on dist-test-slave-5k54"
server_key: "c453813c9f39351f71f40728cd1b8de7"
server_key_iv: "6862d722e742a45e1193268e5b1f089f"
server_key_version: "encryptionkey@0"
I20260808 18:26:40.923401 12085 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:26:40.925822 12100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:40.927066 12085 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:26:40.927197 12085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "5ea358857ebb435b951751d828ba27c9"
format_stamp: "Formatted at 2026-08-08 18:26:40 on dist-test-slave-5k54"
server_key: "c453813c9f39351f71f40728cd1b8de7"
server_key_iv: "6862d722e742a45e1193268e5b1f089f"
server_key_version: "encryptionkey@0"
I20260808 18:26:40.927302 12085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:26:40.939996 12085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:26:40.943094 12085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:26:40.943292 12085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:26:40.950357 12085 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.190:35339
I20260808 18:26:40.950475 12156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.190:35339 every 8 connection(s)
I20260808 18:26:40.951319 12085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:26:40.952476 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 12085
I20260808 18:26:40.952627 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:26:40.952991 23842 external_mini_cluster.cc:1428] Setting key ee79ab16b5131f355bde2d02e731a7cd
I20260808 18:26:40.954517 12157 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:26:40.961306 12157 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ea358857ebb435b951751d828ba27c9: Bootstrap starting.
Aug 08 18:26:40 dist-test-slave-5k54 krb5kdc[12072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:26:40.963696 12157 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ea358857ebb435b951751d828ba27c9: Neither blocks nor log segments found. Creating new log.
I20260808 18:26:40.964653 12157 log.cc:824] T 00000000000000000000000000000000 P 5ea358857ebb435b951751d828ba27c9: Log is configured to *not* fsync() on all Append() calls
I20260808 18:26:40.966866 12157 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ea358857ebb435b951751d828ba27c9: No bootstrap required, opened a new log
I20260808 18:26:40.968211 12160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:40.954499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54816 (local address 127.23.72.190:35339)
0808 18:26:40.954941 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:40.954967 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:40.955008 (+ 41us) server_negotiation.cc:407] Connection header received
0808 18:26:40.955884 (+ 876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:40.955904 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:40.956319 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:40.956664 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:40.957354 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:40.958886 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:40.959824 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:40.960543 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:40.960704 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:40.963236 (+ 2532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:40.963242 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:40.963258 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:26:40.963288 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:40.965342 (+ 2054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:40.965776 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:40.965782 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:40.965788 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:40.965921 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:40.966182 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:40.966185 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:40.966187 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:40.966659 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:40.966862 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:40.967162 (+ 300us) server_negotiation.cc:299] Negotiation successful
0808 18:26:40.967503 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":156,"threads_started":1}
I20260808 18:26:40.970526 12157 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ea358857ebb435b951751d828ba27c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ea358857ebb435b951751d828ba27c9" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 35339 } }
I20260808 18:26:40.970760 12157 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5ea358857ebb435b951751d828ba27c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:26:40.970842 12157 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ea358857ebb435b951751d828ba27c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ea358857ebb435b951751d828ba27c9, State: Initialized, Role: FOLLOWER
I20260808 18:26:40.971280 12157 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ea358857ebb435b951751d828ba27c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ea358857ebb435b951751d828ba27c9" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 35339 } }
I20260808 18:26:40.971467 12157 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5ea358857ebb435b951751d828ba27c9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:26:40.971539 12157 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ea358857ebb435b951751d828ba27c9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:26:40.971639 12157 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ea358857ebb435b951751d828ba27c9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:26:40.972613 12157 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ea358857ebb435b951751d828ba27c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ea358857ebb435b951751d828ba27c9" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 35339 } }
I20260808 18:26:40.972939 12157 leader_election.cc:302] T 00000000000000000000000000000000 P 5ea358857ebb435b951751d828ba27c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5ea358857ebb435b951751d828ba27c9; no voters:
I20260808 18:26:40.973235 12157 leader_election.cc:288] T 00000000000000000000000000000000 P 5ea358857ebb435b951751d828ba27c9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:26:40.973373 12162 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ea358857ebb435b951751d828ba27c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:26:40.973606 12162 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5ea358857ebb435b951751d828ba27c9 [term 1 LEADER]: Becoming Leader. State: Replica: 5ea358857ebb435b951751d828ba27c9, State: Running, Role: LEADER
I20260808 18:26:40.973950 12162 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5ea358857ebb435b951751d828ba27c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ea358857ebb435b951751d828ba27c9" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 35339 } }
I20260808 18:26:40.974545 12157 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ea358857ebb435b951751d828ba27c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:26:40.975788 12164 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ea358857ebb435b951751d828ba27c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ea358857ebb435b951751d828ba27c9. Latest consensus state: current_term: 1 leader_uuid: "5ea358857ebb435b951751d828ba27c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ea358857ebb435b951751d828ba27c9" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 35339 } } }
I20260808 18:26:40.975756 12163 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ea358857ebb435b951751d828ba27c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5ea358857ebb435b951751d828ba27c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ea358857ebb435b951751d828ba27c9" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 35339 } } }
I20260808 18:26:40.975900 12163 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ea358857ebb435b951751d828ba27c9 [sys.catalog]: This master's current role is: LEADER
I20260808 18:26:40.975900 12164 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ea358857ebb435b951751d828ba27c9 [sys.catalog]: This master's current role is: LEADER
I20260808 18:26:40.976868 12171 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:26:40.980335 12171 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:26:40.985221 12171 catalog_manager.cc:1347] Generated new cluster ID: 6216cc2c829d49bc9528e2a4f325d2c8
I20260808 18:26:40.985284 12171 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:26:40.995787 12171 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:26:40.996646 12171 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:26:41.003156 12171 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5ea358857ebb435b951751d828ba27c9: Generated new TSK 0
I20260808 18:26:41.003898 12171 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.72.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab.
Entry for principal kudu/127.23.72.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab.
I20260808 18:26:41.038798 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.129:0
--local_ip_for_outbound_sockets=127.23.72.129
--webserver_interface=127.23.72.129
--webserver_port=0
--tserver_master_addrs=127.23.72.190:35339
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab
--principal=kudu/127.23.72.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:40433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:26:41.151574 12185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:26:41.151854 12185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:26:41.151925 12185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:26:41.155907 12185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:26:41.155983 12185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:26:41.156068 12185 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.129
I20260808 18:26:41.161147 12185 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:40433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.129.keytab
--principal=kudu/127.23.72.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.72.129
--webserver_port=0
--tserver_master_addrs=127.23.72.190:35339
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12185
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.129
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:26:41.162156 12185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:26:41.163146 12185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:26:41.172665 12193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:41.172679 12191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:41.172688 12190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:41.172731 12185 server_base.cc:1034] running on GCE node
I20260808 18:26:41.173287 12185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:26:41.173936 12185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:26:41.175119 12185 hybrid_clock.cc:648] HybridClock initialized: now 1786213601175107 us; error 27 us; skew 500 ppm
Aug 08 18:26:41 dist-test-slave-5k54 krb5kdc[12072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213601, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:26:41.178050 12185 init.cc:377] Logged in from keytab as kudu/127.23.72.129@KRBTEST.COM (short username kudu)
I20260808 18:26:41.178922 12185 webserver.cc:460] Webserver started at http://127.23.72.129:34443/ using document root <none> and password file <none>
I20260808 18:26:41.179548 12185 fs_manager.cc:359] Metadata directory not provided
I20260808 18:26:41.179639 12185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:26:41.179873 12185 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:26:41.181869 12185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-0/data/instance:
uuid: "5211a6d11adb4533a294f1306fbce8f0"
format_stamp: "Formatted at 2026-08-08 18:26:41 on dist-test-slave-5k54"
server_key: "bded4f90fe0fb7835bccc9ebb29873c1"
server_key_iv: "9bbf8ac997fffaabadb0b02cb942ae78"
server_key_version: "encryptionkey@0"
I20260808 18:26:41.182443 12185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance:
uuid: "5211a6d11adb4533a294f1306fbce8f0"
format_stamp: "Formatted at 2026-08-08 18:26:41 on dist-test-slave-5k54"
server_key: "bded4f90fe0fb7835bccc9ebb29873c1"
server_key_iv: "9bbf8ac997fffaabadb0b02cb942ae78"
server_key_version: "encryptionkey@0"
I20260808 18:26:41.186120 12185 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260808 18:26:41.188550 12200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:41.189750 12185 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:26:41.189893 12185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-0/wal
uuid: "5211a6d11adb4533a294f1306fbce8f0"
format_stamp: "Formatted at 2026-08-08 18:26:41 on dist-test-slave-5k54"
server_key: "bded4f90fe0fb7835bccc9ebb29873c1"
server_key_iv: "9bbf8ac997fffaabadb0b02cb942ae78"
server_key_version: "encryptionkey@0"
I20260808 18:26:41.190002 12185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:26:41.221267 12185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:26:41.224434 12185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:26:41.224653 12185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:26:41.225275 12185 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:26:41.226116 12185 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:26:41.226186 12185 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:41.226262 12185 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:26:41.226313 12185 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:41.235756 12185 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.129:38211
I20260808 18:26:41.235809 12320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.129:38211 every 8 connection(s)
I20260808 18:26:41.236757 12185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
I20260808 18:26:41.245965 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 12185
I20260808 18:26:41.246125 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance
Aug 08 18:26:41 dist-test-slave-5k54 krb5kdc[12072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213601, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.129@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:26:41.246503 23842 external_mini_cluster.cc:1428] Setting key 97c765bad4259da971e6e3c198b259eb
I20260808 18:26:41.251302 12160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:41.238502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:59387 (local address 127.23.72.190:35339)
0808 18:26:41.238650 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:41.238653 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:41.239396 (+ 743us) server_negotiation.cc:407] Connection header received
0808 18:26:41.240729 (+ 1333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:41.240732 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:41.240797 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:41.240899 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:41.242091 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:41.242634 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:41.243807 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:41.244049 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:41.244163 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:41.247021 (+ 2858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:41.247025 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:41.247028 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:26:41.247058 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:41.248887 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:41.249543 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:41.249546 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:41.249547 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:41.249628 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:41.250036 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:41.250039 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:41.250040 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:41.250268 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:41.250400 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:41.250963 (+ 563us) server_negotiation.cc:299] Negotiation successful
0808 18:26:41.251109 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 18:26:41.252161 12325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:41.238802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:35339 (local address 127.23.72.129:59387)
0808 18:26:41.239243 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:41.239284 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:41.240512 (+ 1228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:41.241098 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:41.241105 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:41.241459 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:26:41.241916 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:41.241947 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:41.242764 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:41.242768 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:41.243667 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:41.243676 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:41.244205 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:41.244209 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:41.244382 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:41.244990 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:26:41.245006 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:26:41.246778 (+ 1772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:26:41.249017 (+ 2239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:26:41.249026 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:26:41.249044 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:26:41.249438 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:26:41.249721 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:26:41.249727 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:26:41.249732 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:26:41.249926 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:26:41.250402 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:26:41.250410 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:26:41.250674 (+ 264us) client_negotiation.cc:769] Sending connection context
0808 18:26:41.250890 (+ 216us) client_negotiation.cc:240] Negotiation successful
0808 18:26:41.251243 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":112,"threads_started":1}
I20260808 18:26:41.253505 12323 heartbeater.cc:347] Connected to a master server at 127.23.72.190:35339
I20260808 18:26:41.253767 12323 heartbeater.cc:464] Registering TS with master...
I20260808 18:26:41.254400 12323 heartbeater.cc:511] Master 127.23.72.190:35339 requested a full tablet report, sending...
I20260808 18:26:41.256279 12119 ts_manager.cc:194] Registered new tserver with Master: 5211a6d11adb4533a294f1306fbce8f0 (127.23.72.129:38211)
WARNING: no policy specified for kudu/127.23.72.130@KRBTEST.COM; defaulting to no policy
I20260808 18:26:41.257995 12119 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.129@KRBTEST.COM'} at 127.23.72.129:59387
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab.
Entry for principal kudu/127.23.72.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab.
I20260808 18:26:41.276396 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.130:0
--local_ip_for_outbound_sockets=127.23.72.130
--webserver_interface=127.23.72.130
--webserver_port=0
--tserver_master_addrs=127.23.72.190:35339
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab
--principal=kudu/127.23.72.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:40433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:26:41.388631 12328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:26:41.388890 12328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:26:41.388969 12328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:26:41.393067 12328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:26:41.393126 12328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:26:41.393211 12328 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.130
I20260808 18:26:41.398550 12328 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:40433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.130.keytab
--principal=kudu/127.23.72.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.72.130
--webserver_port=0
--tserver_master_addrs=127.23.72.190:35339
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.130
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:26:41.399519 12328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:26:41.400453 12328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:26:41.410066 12333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:41.410266 12328 server_base.cc:1034] running on GCE node
W20260808 18:26:41.410105 12334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:41.410056 12336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:41.410812 12328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:26:41.411367 12328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:26:41.412576 12328 hybrid_clock.cc:648] HybridClock initialized: now 1786213601412556 us; error 51 us; skew 500 ppm
Aug 08 18:26:41 dist-test-slave-5k54 krb5kdc[12072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213601, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:26:41.415302 12328 init.cc:377] Logged in from keytab as kudu/127.23.72.130@KRBTEST.COM (short username kudu)
I20260808 18:26:41.416230 12328 webserver.cc:460] Webserver started at http://127.23.72.130:37349/ using document root <none> and password file <none>
I20260808 18:26:41.416790 12328 fs_manager.cc:359] Metadata directory not provided
I20260808 18:26:41.416913 12328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:26:41.417143 12328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:26:41.419142 12328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-1/data/instance:
uuid: "ca8d18fa287d4ed9bd77936ccd335850"
format_stamp: "Formatted at 2026-08-08 18:26:41 on dist-test-slave-5k54"
server_key: "3b3142d88ac6398e35355cbeec80a3b1"
server_key_iv: "764d45508db0e23dfe8122fe56905dc5"
server_key_version: "encryptionkey@0"
I20260808 18:26:41.419734 12328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance:
uuid: "ca8d18fa287d4ed9bd77936ccd335850"
format_stamp: "Formatted at 2026-08-08 18:26:41 on dist-test-slave-5k54"
server_key: "3b3142d88ac6398e35355cbeec80a3b1"
server_key_iv: "764d45508db0e23dfe8122fe56905dc5"
server_key_version: "encryptionkey@0"
I20260808 18:26:41.423516 12328 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260808 18:26:41.426002 12343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:41.427217 12328 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 18:26:41.427377 12328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-1/wal
uuid: "ca8d18fa287d4ed9bd77936ccd335850"
format_stamp: "Formatted at 2026-08-08 18:26:41 on dist-test-slave-5k54"
server_key: "3b3142d88ac6398e35355cbeec80a3b1"
server_key_iv: "764d45508db0e23dfe8122fe56905dc5"
server_key_version: "encryptionkey@0"
I20260808 18:26:41.427559 12328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:26:41.449334 12328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:26:41.452397 12328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:26:41.452632 12328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:26:41.453255 12328 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:26:41.454093 12328 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:26:41.454166 12328 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:41.454241 12328 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:26:41.454293 12328 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:41.463393 12328 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.130:34585
I20260808 18:26:41.463477 12460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.130:34585 every 8 connection(s)
I20260808 18:26:41.464355 12328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
I20260808 18:26:41.473423 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 12328
I20260808 18:26:41.473579 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance
I20260808 18:26:41.473914 23842 external_mini_cluster.cc:1428] Setting key 111b68f2a0ec13a41f1f7694c6aa899b
Aug 08 18:26:41 dist-test-slave-5k54 krb5kdc[12072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213601, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.130@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:26:41.479169 12160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:41.466246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:43749 (local address 127.23.72.190:35339)
0808 18:26:41.466391 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:41.466395 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:41.467217 (+ 822us) server_negotiation.cc:407] Connection header received
0808 18:26:41.468615 (+ 1398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:41.468619 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:41.468678 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:41.468784 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:41.470033 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:41.470575 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:41.471833 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:41.472060 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:41.472172 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:41.474942 (+ 2770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:41.474945 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:41.474947 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:26:41.474972 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:41.476808 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:41.477384 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:41.477391 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:41.477395 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:41.477496 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:41.477898 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:41.477904 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:41.477907 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:41.478103 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:41.478162 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:41.478815 (+ 653us) server_negotiation.cc:299] Negotiation successful
0808 18:26:41.478961 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 18:26:41.480034 12463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:41.466631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:35339 (local address 127.23.72.130:43749)
0808 18:26:41.467068 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:41.467110 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:41.468401 (+ 1291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:41.469038 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:41.469045 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:41.469404 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:26:41.469887 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:41.469900 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:41.470707 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:41.470715 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:41.471674 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:41.471683 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:41.472213 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:41.472217 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:41.472387 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:41.472917 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:26:41.472932 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:26:41.474787 (+ 1855us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:26:41.476934 (+ 2147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:26:41.476942 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:26:41.476958 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:26:41.477281 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:26:41.477598 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:26:41.477603 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:26:41.477607 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:26:41.477774 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:26:41.478230 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:26:41.478235 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:26:41.478460 (+ 225us) client_negotiation.cc:769] Sending connection context
0808 18:26:41.478691 (+ 231us) client_negotiation.cc:240] Negotiation successful
0808 18:26:41.478942 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":113,"threads_started":1}
I20260808 18:26:41.481199 12461 heartbeater.cc:347] Connected to a master server at 127.23.72.190:35339
I20260808 18:26:41.481460 12461 heartbeater.cc:464] Registering TS with master...
I20260808 18:26:41.481973 12461 heartbeater.cc:511] Master 127.23.72.190:35339 requested a full tablet report, sending...
I20260808 18:26:41.483016 12119 ts_manager.cc:194] Registered new tserver with Master: ca8d18fa287d4ed9bd77936ccd335850 (127.23.72.130:34585)
I20260808 18:26:41.483661 12119 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.130@KRBTEST.COM'} at 127.23.72.130:43749
WARNING: no policy specified for kudu/127.23.72.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab.
Entry for principal kudu/127.23.72.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab.
I20260808 18:26:41.506851 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.131:0
--local_ip_for_outbound_sockets=127.23.72.131
--webserver_interface=127.23.72.131
--webserver_port=0
--tserver_master_addrs=127.23.72.190:35339
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab
--principal=kudu/127.23.72.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:40433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:26:41.616772 12466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:26:41.617057 12466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:26:41.617133 12466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:26:41.621094 12466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:26:41.621178 12466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:26:41.621264 12466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.131
I20260808 18:26:41.627379 12466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:40433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.131.keytab
--principal=kudu/127.23.72.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.72.131
--webserver_port=0
--tserver_master_addrs=127.23.72.190:35339
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.131
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:26:41.628461 12466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:26:41.629468 12466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:26:41.639045 12474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:41.639084 12476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:41.639084 12473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:41.639303 12466 server_base.cc:1034] running on GCE node
I20260808 18:26:41.639803 12466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:26:41.640436 12466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:26:41.641625 12466 hybrid_clock.cc:648] HybridClock initialized: now 1786213601641609 us; error 29 us; skew 500 ppm
Aug 08 18:26:41 dist-test-slave-5k54 krb5kdc[12072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213601, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:26:41.644474 12466 init.cc:377] Logged in from keytab as kudu/127.23.72.131@KRBTEST.COM (short username kudu)
I20260808 18:26:41.645342 12466 webserver.cc:460] Webserver started at http://127.23.72.131:44097/ using document root <none> and password file <none>
I20260808 18:26:41.645896 12466 fs_manager.cc:359] Metadata directory not provided
I20260808 18:26:41.645988 12466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:26:41.646224 12466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:26:41.648312 12466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-2/data/instance:
uuid: "f6ef84bdc5034338a990eb42dee4b2f0"
format_stamp: "Formatted at 2026-08-08 18:26:41 on dist-test-slave-5k54"
server_key: "935031d0c4aedf8ce80e9e1367fb9776"
server_key_iv: "309c285c1e52249cc7f42afa5788af20"
server_key_version: "encryptionkey@0"
I20260808 18:26:41.648890 12466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance:
uuid: "f6ef84bdc5034338a990eb42dee4b2f0"
format_stamp: "Formatted at 2026-08-08 18:26:41 on dist-test-slave-5k54"
server_key: "935031d0c4aedf8ce80e9e1367fb9776"
server_key_iv: "309c285c1e52249cc7f42afa5788af20"
server_key_version: "encryptionkey@0"
I20260808 18:26:41.652540 12466 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260808 18:26:41.654875 12483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:41.656035 12466 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:26:41.656167 12466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-2/wal
uuid: "f6ef84bdc5034338a990eb42dee4b2f0"
format_stamp: "Formatted at 2026-08-08 18:26:41 on dist-test-slave-5k54"
server_key: "935031d0c4aedf8ce80e9e1367fb9776"
server_key_iv: "309c285c1e52249cc7f42afa5788af20"
server_key_version: "encryptionkey@0"
I20260808 18:26:41.656273 12466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:26:41.664816 12466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:26:41.667459 12466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:26:41.667665 12466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:26:41.668200 12466 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:26:41.668975 12466 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:26:41.669044 12466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:41.669113 12466 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:26:41.669155 12466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:41.678596 12466 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.131:36333
I20260808 18:26:41.678678 12607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.131:36333 every 8 connection(s)
I20260808 18:26:41.679713 12466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
I20260808 18:26:41.683459 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 12466
I20260808 18:26:41.683609 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance
I20260808 18:26:41.683934 23842 external_mini_cluster.cc:1428] Setting key b97a1bfaee84f5a6c224b4394dd1bd5c
Aug 08 18:26:41 dist-test-slave-5k54 krb5kdc[12072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213601, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.131@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:26:41.693444 12160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:41.681803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:53871 (local address 127.23.72.190:35339)
0808 18:26:41.681933 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:41.681944 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:41.682734 (+ 790us) server_negotiation.cc:407] Connection header received
0808 18:26:41.684386 (+ 1652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:41.684389 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:41.684452 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:41.684557 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:41.685808 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:41.686346 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:41.687471 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:41.687709 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:41.687821 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:41.690162 (+ 2341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:41.690165 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:41.690168 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:26:41.690196 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:41.691463 (+ 1267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:41.691960 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:41.691965 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:41.691966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:41.692053 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:41.692383 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:41.692388 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:41.692390 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:41.692568 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:41.692673 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:41.693144 (+ 471us) server_negotiation.cc:299] Negotiation successful
0808 18:26:41.693264 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 18:26:41.694223 12610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:41.682166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:35339 (local address 127.23.72.131:53871)
0808 18:26:41.682582 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:41.682620 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:41.684146 (+ 1526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:41.684729 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:41.684736 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:41.685206 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:26:41.685669 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:41.685683 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:41.686469 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:41.686472 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:41.687330 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:41.687340 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:41.687861 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:41.687865 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:41.688035 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:41.688541 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:26:41.688556 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:26:41.690003 (+ 1447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:26:41.691584 (+ 1581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:26:41.691589 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:26:41.691602 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:26:41.691858 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:26:41.692144 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:26:41.692147 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:26:41.692150 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:26:41.692283 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:26:41.692670 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:26:41.692677 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:26:41.692921 (+ 244us) client_negotiation.cc:769] Sending connection context
0808 18:26:41.693101 (+ 180us) client_negotiation.cc:240] Negotiation successful
0808 18:26:41.693403 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":108,"threads_started":1}
I20260808 18:26:41.695279 12608 heartbeater.cc:347] Connected to a master server at 127.23.72.190:35339
I20260808 18:26:41.695534 12608 heartbeater.cc:464] Registering TS with master...
I20260808 18:26:41.695966 12608 heartbeater.cc:511] Master 127.23.72.190:35339 requested a full tablet report, sending...
I20260808 18:26:41.696929 12119 ts_manager.cc:194] Registered new tserver with Master: f6ef84bdc5034338a990eb42dee4b2f0 (127.23.72.131:36333)
I20260808 18:26:41.697628 12119 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.72.131@KRBTEST.COM'} at 127.23.72.131:53871
I20260808 18:26:41.697793 23842 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:26:41.706732 12160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:41.699880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54818 (local address 127.23.72.190:35339)
0808 18:26:41.700028 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:41.700033 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:41.700186 (+ 153us) server_negotiation.cc:407] Connection header received
0808 18:26:41.700264 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:41.700266 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:41.700301 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:41.700364 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:41.701024 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:41.701490 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:41.702293 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:41.702493 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:41.702564 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:41.703838 (+ 1274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:41.703840 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:41.703842 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:26:41.703862 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:41.705267 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:41.705631 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:41.705633 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:41.705634 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:41.705705 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:41.705962 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:41.705964 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:41.705965 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:41.706153 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:41.706248 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:41.706441 (+ 193us) server_negotiation.cc:299] Negotiation successful
0808 18:26:41.706527 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 18:26:41.709576 12119 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54818:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:26:41.712544 12119 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:26:41.726532 12620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:41.720609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.131:36333 (local address 127.0.0.1:46406)
0808 18:26:41.721086 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:41.721126 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:41.721249 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:41.722002 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:41.722007 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:41.722044 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:26:41.722260 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:41.722268 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:41.723714 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:41.723717 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:41.724938 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:41.724945 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:41.725968 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:41.725972 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:41.726110 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:41.726143 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 18:26:41.726216 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 18:26:41.726373 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"mutex_wait_us":111,"spinlock_wait_cycles":12672,"thread_start_us":102,"threads_started":1}
I20260808 18:26:41.726567 12621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:41.720609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52902 (local address 127.23.72.129:38211)
0808 18:26:41.720905 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:41.720932 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:41.721401 (+ 469us) server_negotiation.cc:407] Connection header received
0808 18:26:41.721486 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:41.721491 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:41.721698 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:41.721855 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:26:41.722433 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:41.723315 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:41.725110 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:41.725761 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:41.725887 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:41.726038 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:41.726289 (+ 251us) server_negotiation.cc:299] Negotiation successful
0808 18:26:41.726406 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":112,"threads_started":1}
I20260808 18:26:41.726748 12625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:41.721633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36848 (local address 127.23.72.130:34585)
0808 18:26:41.722025 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:41.722042 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:41.722306 (+ 264us) server_negotiation.cc:407] Connection header received
0808 18:26:41.722462 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:41.722467 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:41.722632 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:41.722774 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:26:41.723008 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:41.723978 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:41.725383 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:41.726215 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:41.726322 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:41.726459 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:41.726528 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 18:26:41.726623 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":93,"threads_started":1}
I20260808 18:26:41.727012 12623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:41.721170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46406 (local address 127.23.72.131:36333)
0808 18:26:41.721536 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:41.721549 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:41.721571 (+ 22us) server_negotiation.cc:407] Connection header received
0808 18:26:41.721638 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:41.721642 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:41.721825 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:41.722116 (+ 291us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:26:41.722389 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:41.723579 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:41.725072 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:41.725816 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:41.726630 (+ 814us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:41.726752 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:41.726800 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 18:26:41.726895 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":160,"threads_started":1}
I20260808 18:26:41.727306 12624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:41.721229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.130:34585 (local address 127.0.0.1:36848)
0808 18:26:41.722205 (+ 976us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:41.722219 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:41.722307 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:41.722786 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:41.722789 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:41.722808 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:26:41.722891 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:41.722896 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:41.724106 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:41.724112 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:41.725246 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:41.725257 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:41.726339 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:41.726342 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:41.726419 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:41.726439 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 18:26:41.726489 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 18:26:41.726545 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":921,"thread_start_us":99,"threads_started":1}
I20260808 18:26:41.727808 12619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:41.720417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:38211 (local address 127.0.0.1:52902)
0808 18:26:41.721063 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:41.721112 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:41.721249 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:41.722015 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:41.722017 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:41.722045 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:26:41.722335 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:41.722344 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:41.723469 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:41.723473 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:41.724926 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:41.724937 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:41.725889 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:41.725893 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:41.726077 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:41.726137 (+ 60us) client_negotiation.cc:769] Sending connection context
0808 18:26:41.726252 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 18:26:41.727685 (+ 1433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"mutex_wait_us":1599,"spinlock_wait_cycles":12288,"thread_start_us":78,"threads_started":1}
I20260808 18:26:41.729770 12536 tablet_service.cc:1467] Processing CreateTablet for tablet 6ca94d8cee0e4c56b2a2888a81d2b291 (DEFAULT_TABLE table=test-table [id=5bbd51b016324ae9bf93e780d1570c21]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:26:41.729892 12392 tablet_service.cc:1467] Processing CreateTablet for tablet 6ca94d8cee0e4c56b2a2888a81d2b291 (DEFAULT_TABLE table=test-table [id=5bbd51b016324ae9bf93e780d1570c21]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:26:41.730736 12536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ca94d8cee0e4c56b2a2888a81d2b291. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:26:41.730414 12251 tablet_service.cc:1467] Processing CreateTablet for tablet 6ca94d8cee0e4c56b2a2888a81d2b291 (DEFAULT_TABLE table=test-table [id=5bbd51b016324ae9bf93e780d1570c21]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:26:41.730904 12392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ca94d8cee0e4c56b2a2888a81d2b291. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:26:41.731293 12251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ca94d8cee0e4c56b2a2888a81d2b291. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:26:41.738204 12626 tablet_bootstrap.cc:492] T 6ca94d8cee0e4c56b2a2888a81d2b291 P f6ef84bdc5034338a990eb42dee4b2f0: Bootstrap starting.
I20260808 18:26:41.738595 12627 tablet_bootstrap.cc:492] T 6ca94d8cee0e4c56b2a2888a81d2b291 P 5211a6d11adb4533a294f1306fbce8f0: Bootstrap starting.
I20260808 18:26:41.738629 12628 tablet_bootstrap.cc:492] T 6ca94d8cee0e4c56b2a2888a81d2b291 P ca8d18fa287d4ed9bd77936ccd335850: Bootstrap starting.
I20260808 18:26:41.740460 12627 tablet_bootstrap.cc:654] T 6ca94d8cee0e4c56b2a2888a81d2b291 P 5211a6d11adb4533a294f1306fbce8f0: Neither blocks nor log segments found. Creating new log.
I20260808 18:26:41.740600 12628 tablet_bootstrap.cc:654] T 6ca94d8cee0e4c56b2a2888a81d2b291 P ca8d18fa287d4ed9bd77936ccd335850: Neither blocks nor log segments found. Creating new log.
I20260808 18:26:41.741387 12626 tablet_bootstrap.cc:654] T 6ca94d8cee0e4c56b2a2888a81d2b291 P f6ef84bdc5034338a990eb42dee4b2f0: Neither blocks nor log segments found. Creating new log.
I20260808 18:26:41.741601 12627 log.cc:824] T 6ca94d8cee0e4c56b2a2888a81d2b291 P 5211a6d11adb4533a294f1306fbce8f0: Log is configured to *not* fsync() on all Append() calls
I20260808 18:26:41.741868 12628 log.cc:824] T 6ca94d8cee0e4c56b2a2888a81d2b291 P ca8d18fa287d4ed9bd77936ccd335850: Log is configured to *not* fsync() on all Append() calls
I20260808 18:26:41.742496 12626 log.cc:824] T 6ca94d8cee0e4c56b2a2888a81d2b291 P f6ef84bdc5034338a990eb42dee4b2f0: Log is configured to *not* fsync() on all Append() calls
I20260808 18:26:41.743731 12627 tablet_bootstrap.cc:492] T 6ca94d8cee0e4c56b2a2888a81d2b291 P 5211a6d11adb4533a294f1306fbce8f0: No bootstrap required, opened a new log
I20260808 18:26:41.743943 12627 ts_tablet_manager.cc:1397] T 6ca94d8cee0e4c56b2a2888a81d2b291 P 5211a6d11adb4533a294f1306fbce8f0: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260808 18:26:41.744369 12626 tablet_bootstrap.cc:492] T 6ca94d8cee0e4c56b2a2888a81d2b291 P f6ef84bdc5034338a990eb42dee4b2f0: No bootstrap required, opened a new log
I20260808 18:26:41.744556 12626 ts_tablet_manager.cc:1397] T 6ca94d8cee0e4c56b2a2888a81d2b291 P f6ef84bdc5034338a990eb42dee4b2f0: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 18:26:41.745054 12628 tablet_bootstrap.cc:492] T 6ca94d8cee0e4c56b2a2888a81d2b291 P ca8d18fa287d4ed9bd77936ccd335850: No bootstrap required, opened a new log
I20260808 18:26:41.745222 12628 ts_tablet_manager.cc:1397] T 6ca94d8cee0e4c56b2a2888a81d2b291 P ca8d18fa287d4ed9bd77936ccd335850: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260808 18:26:41.747347 12626 raft_consensus.cc:348] T 6ca94d8cee0e4c56b2a2888a81d2b291 P f6ef84bdc5034338a990eb42dee4b2f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5211a6d11adb4533a294f1306fbce8f0" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 38211 } } peers { permanent_uuid: "f6ef84bdc5034338a990eb42dee4b2f0" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 36333 } } peers { permanent_uuid: "ca8d18fa287d4ed9bd77936ccd335850" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 34585 } }
I20260808 18:26:41.747633 12626 raft_consensus.cc:374] T 6ca94d8cee0e4c56b2a2888a81d2b291 P f6ef84bdc5034338a990eb42dee4b2f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:26:41.747717 12626 raft_consensus.cc:729] T 6ca94d8cee0e4c56b2a2888a81d2b291 P f6ef84bdc5034338a990eb42dee4b2f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6ef84bdc5034338a990eb42dee4b2f0, State: Initialized, Role: FOLLOWER
I20260808 18:26:41.747857 12627 raft_consensus.cc:348] T 6ca94d8cee0e4c56b2a2888a81d2b291 P 5211a6d11adb4533a294f1306fbce8f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5211a6d11adb4533a294f1306fbce8f0" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 38211 } } peers { permanent_uuid: "f6ef84bdc5034338a990eb42dee4b2f0" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 36333 } } peers { permanent_uuid: "ca8d18fa287d4ed9bd77936ccd335850" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 34585 } }
I20260808 18:26:41.748117 12627 raft_consensus.cc:374] T 6ca94d8cee0e4c56b2a2888a81d2b291 P 5211a6d11adb4533a294f1306fbce8f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:26:41.748222 12627 raft_consensus.cc:729] T 6ca94d8cee0e4c56b2a2888a81d2b291 P 5211a6d11adb4533a294f1306fbce8f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5211a6d11adb4533a294f1306fbce8f0, State: Initialized, Role: FOLLOWER
I20260808 18:26:41.748217 12626 consensus_queue.cc:260] T 6ca94d8cee0e4c56b2a2888a81d2b291 P f6ef84bdc5034338a990eb42dee4b2f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5211a6d11adb4533a294f1306fbce8f0" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 38211 } } peers { permanent_uuid: "f6ef84bdc5034338a990eb42dee4b2f0" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 36333 } } peers { permanent_uuid: "ca8d18fa287d4ed9bd77936ccd335850" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 34585 } }
I20260808 18:26:41.748695 12627 consensus_queue.cc:260] T 6ca94d8cee0e4c56b2a2888a81d2b291 P 5211a6d11adb4533a294f1306fbce8f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5211a6d11adb4533a294f1306fbce8f0" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 38211 } } peers { permanent_uuid: "f6ef84bdc5034338a990eb42dee4b2f0" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 36333 } } peers { permanent_uuid: "ca8d18fa287d4ed9bd77936ccd335850" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 34585 } }
I20260808 18:26:41.748906 12608 heartbeater.cc:503] Master 127.23.72.190:35339 was elected leader, sending a full tablet report...
I20260808 18:26:41.749019 12628 raft_consensus.cc:348] T 6ca94d8cee0e4c56b2a2888a81d2b291 P ca8d18fa287d4ed9bd77936ccd335850 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5211a6d11adb4533a294f1306fbce8f0" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 38211 } } peers { permanent_uuid: "f6ef84bdc5034338a990eb42dee4b2f0" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 36333 } } peers { permanent_uuid: "ca8d18fa287d4ed9bd77936ccd335850" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 34585 } }
I20260808 18:26:41.749313 12628 raft_consensus.cc:374] T 6ca94d8cee0e4c56b2a2888a81d2b291 P ca8d18fa287d4ed9bd77936ccd335850 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:26:41.749403 12628 raft_consensus.cc:729] T 6ca94d8cee0e4c56b2a2888a81d2b291 P ca8d18fa287d4ed9bd77936ccd335850 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca8d18fa287d4ed9bd77936ccd335850, State: Initialized, Role: FOLLOWER
I20260808 18:26:41.749830 12627 ts_tablet_manager.cc:1428] T 6ca94d8cee0e4c56b2a2888a81d2b291 P 5211a6d11adb4533a294f1306fbce8f0: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 18:26:41.749883 12628 consensus_queue.cc:260] T 6ca94d8cee0e4c56b2a2888a81d2b291 P ca8d18fa287d4ed9bd77936ccd335850 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5211a6d11adb4533a294f1306fbce8f0" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 38211 } } peers { permanent_uuid: "f6ef84bdc5034338a990eb42dee4b2f0" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 36333 } } peers { permanent_uuid: "ca8d18fa287d4ed9bd77936ccd335850" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 34585 } }
I20260808 18:26:41.749260 12626 ts_tablet_manager.cc:1428] T 6ca94d8cee0e4c56b2a2888a81d2b291 P f6ef84bdc5034338a990eb42dee4b2f0: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 18:26:41.750283 12323 heartbeater.cc:503] Master 127.23.72.190:35339 was elected leader, sending a full tablet report...
I20260808 18:26:41.750833 12628 ts_tablet_manager.cc:1428] T 6ca94d8cee0e4c56b2a2888a81d2b291 P ca8d18fa287d4ed9bd77936ccd335850: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 18:26:41.751166 12461 heartbeater.cc:503] Master 127.23.72.190:35339 was elected leader, sending a full tablet report...
I20260808 18:26:41.798115 12635 raft_consensus.cc:482] T 6ca94d8cee0e4c56b2a2888a81d2b291 P ca8d18fa287d4ed9bd77936ccd335850 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:26:41.798369 12635 raft_consensus.cc:504] T 6ca94d8cee0e4c56b2a2888a81d2b291 P ca8d18fa287d4ed9bd77936ccd335850 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5211a6d11adb4533a294f1306fbce8f0" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 38211 } } peers { permanent_uuid: "f6ef84bdc5034338a990eb42dee4b2f0" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 36333 } } peers { permanent_uuid: "ca8d18fa287d4ed9bd77936ccd335850" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 34585 } }
I20260808 18:26:41.799602 12635 leader_election.cc:288] T 6ca94d8cee0e4c56b2a2888a81d2b291 P ca8d18fa287d4ed9bd77936ccd335850 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5211a6d11adb4533a294f1306fbce8f0 (127.23.72.129:38211), f6ef84bdc5034338a990eb42dee4b2f0 (127.23.72.131:36333)
I20260808 18:26:41.803324 12621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:41.799859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:41691 (local address 127.23.72.129:38211)
0808 18:26:41.800213 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:41.800219 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:41.800234 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:26:41.800288 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:41.800291 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:41.800364 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:41.800433 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:26:41.800766 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:41.801278 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:41.802350 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:41.802734 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:41.802832 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:41.802919 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:41.803117 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 18:26:41.803194 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257}
I20260808 18:26:41.803354 12623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:41.799959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:32989 (local address 127.23.72.131:36333)
0808 18:26:41.800118 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:41.800123 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:41.800259 (+ 136us) server_negotiation.cc:407] Connection header received
0808 18:26:41.800358 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:41.800360 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:41.800402 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:41.800505 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:26:41.800823 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:41.801297 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:41.802375 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:41.802762 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:41.802904 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:41.802958 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:41.803172 (+ 214us) server_negotiation.cc:299] Negotiation successful
0808 18:26:41.803249 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 18:26:41.803342 12463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:41.799776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:38211 (local address 127.23.72.130:41691)
0808 18:26:41.800034 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:41.800051 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:41.800146 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:41.800474 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:41.800478 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:41.800498 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:26:41.800636 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:41.800643 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:41.801393 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:41.801396 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:41.802212 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:41.802221 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:41.802863 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:41.802866 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:41.802984 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:41.803006 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 18:26:41.803081 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 18:26:41.803171 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":5}
I20260808 18:26:41.803356 12636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:41.799831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.131:36333 (local address 127.23.72.130:32989)
0808 18:26:41.800156 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:41.800170 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:41.800277 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:41.800540 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:41.800545 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:41.800562 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:26:41.800684 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:41.800688 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:41.801430 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:41.801434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:41.802221 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:41.802229 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:41.802927 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:41.802932 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:41.803025 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:41.803041 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 18:26:41.803139 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 18:26:41.803198 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":26,"thread_start_us":147,"threads_started":1}
I20260808 18:26:41.804186 12274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ca94d8cee0e4c56b2a2888a81d2b291" candidate_uuid: "ca8d18fa287d4ed9bd77936ccd335850" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5211a6d11adb4533a294f1306fbce8f0" is_pre_election: true
I20260808 18:26:41.804189 12557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ca94d8cee0e4c56b2a2888a81d2b291" candidate_uuid: "ca8d18fa287d4ed9bd77936ccd335850" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6ef84bdc5034338a990eb42dee4b2f0" is_pre_election: true
I20260808 18:26:41.804497 12274 raft_consensus.cc:2454] T 6ca94d8cee0e4c56b2a2888a81d2b291 P 5211a6d11adb4533a294f1306fbce8f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca8d18fa287d4ed9bd77936ccd335850 in term 0.
I20260808 18:26:41.804497 12557 raft_consensus.cc:2454] T 6ca94d8cee0e4c56b2a2888a81d2b291 P f6ef84bdc5034338a990eb42dee4b2f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca8d18fa287d4ed9bd77936ccd335850 in term 0.
I20260808 18:26:41.805013 12344 leader_election.cc:302] T 6ca94d8cee0e4c56b2a2888a81d2b291 P ca8d18fa287d4ed9bd77936ccd335850 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca8d18fa287d4ed9bd77936ccd335850, f6ef84bdc5034338a990eb42dee4b2f0; no voters:
I20260808 18:26:41.805310 12635 raft_consensus.cc:2781] T 6ca94d8cee0e4c56b2a2888a81d2b291 P ca8d18fa287d4ed9bd77936ccd335850 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:26:41.805408 12635 raft_consensus.cc:482] T 6ca94d8cee0e4c56b2a2888a81d2b291 P ca8d18fa287d4ed9bd77936ccd335850 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:26:41.805471 12635 raft_consensus.cc:3037] T 6ca94d8cee0e4c56b2a2888a81d2b291 P ca8d18fa287d4ed9bd77936ccd335850 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:26:41.806528 12635 raft_consensus.cc:504] T 6ca94d8cee0e4c56b2a2888a81d2b291 P ca8d18fa287d4ed9bd77936ccd335850 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5211a6d11adb4533a294f1306fbce8f0" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 38211 } } peers { permanent_uuid: "f6ef84bdc5034338a990eb42dee4b2f0" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 36333 } } peers { permanent_uuid: "ca8d18fa287d4ed9bd77936ccd335850" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 34585 } }
I20260808 18:26:41.806872 12635 leader_election.cc:288] T 6ca94d8cee0e4c56b2a2888a81d2b291 P ca8d18fa287d4ed9bd77936ccd335850 [CANDIDATE]: Term 1 election: Requested vote from peers 5211a6d11adb4533a294f1306fbce8f0 (127.23.72.129:38211), f6ef84bdc5034338a990eb42dee4b2f0 (127.23.72.131:36333)
I20260808 18:26:41.807206 12557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ca94d8cee0e4c56b2a2888a81d2b291" candidate_uuid: "ca8d18fa287d4ed9bd77936ccd335850" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6ef84bdc5034338a990eb42dee4b2f0"
I20260808 18:26:41.807210 12274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ca94d8cee0e4c56b2a2888a81d2b291" candidate_uuid: "ca8d18fa287d4ed9bd77936ccd335850" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5211a6d11adb4533a294f1306fbce8f0"
I20260808 18:26:41.807344 12557 raft_consensus.cc:3037] T 6ca94d8cee0e4c56b2a2888a81d2b291 P f6ef84bdc5034338a990eb42dee4b2f0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:26:41.807344 12274 raft_consensus.cc:3037] T 6ca94d8cee0e4c56b2a2888a81d2b291 P 5211a6d11adb4533a294f1306fbce8f0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:26:41.808666 12274 raft_consensus.cc:2454] T 6ca94d8cee0e4c56b2a2888a81d2b291 P 5211a6d11adb4533a294f1306fbce8f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca8d18fa287d4ed9bd77936ccd335850 in term 1.
I20260808 18:26:41.808666 12557 raft_consensus.cc:2454] T 6ca94d8cee0e4c56b2a2888a81d2b291 P f6ef84bdc5034338a990eb42dee4b2f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca8d18fa287d4ed9bd77936ccd335850 in term 1.
I20260808 18:26:41.808996 12346 leader_election.cc:302] T 6ca94d8cee0e4c56b2a2888a81d2b291 P ca8d18fa287d4ed9bd77936ccd335850 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5211a6d11adb4533a294f1306fbce8f0, ca8d18fa287d4ed9bd77936ccd335850; no voters:
I20260808 18:26:41.809195 12635 raft_consensus.cc:2781] T 6ca94d8cee0e4c56b2a2888a81d2b291 P ca8d18fa287d4ed9bd77936ccd335850 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:26:41.809422 12635 raft_consensus.cc:686] T 6ca94d8cee0e4c56b2a2888a81d2b291 P ca8d18fa287d4ed9bd77936ccd335850 [term 1 LEADER]: Becoming Leader. State: Replica: ca8d18fa287d4ed9bd77936ccd335850, State: Running, Role: LEADER
I20260808 18:26:41.809751 12635 consensus_queue.cc:237] T 6ca94d8cee0e4c56b2a2888a81d2b291 P ca8d18fa287d4ed9bd77936ccd335850 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5211a6d11adb4533a294f1306fbce8f0" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 38211 } } peers { permanent_uuid: "f6ef84bdc5034338a990eb42dee4b2f0" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 36333 } } peers { permanent_uuid: "ca8d18fa287d4ed9bd77936ccd335850" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 34585 } }
I20260808 18:26:41.813092 12117 catalog_manager.cc:5495] T 6ca94d8cee0e4c56b2a2888a81d2b291 P ca8d18fa287d4ed9bd77936ccd335850 reported cstate change: term changed from 0 to 1, leader changed from <none> to ca8d18fa287d4ed9bd77936ccd335850 (127.23.72.130). New cstate: current_term: 1 leader_uuid: "ca8d18fa287d4ed9bd77936ccd335850" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5211a6d11adb4533a294f1306fbce8f0" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 38211 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6ef84bdc5034338a990eb42dee4b2f0" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 36333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca8d18fa287d4ed9bd77936ccd335850" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 34585 } health_report { overall_health: HEALTHY } } }
I20260808 18:26:41.824165 12625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:41.820622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36862 (local address 127.23.72.130:34585)
0808 18:26:41.820778 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:41.820789 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:41.820801 (+ 12us) server_negotiation.cc:407] Connection header received
0808 18:26:41.820861 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:41.820863 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:41.820913 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:41.821041 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:26:41.821297 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:41.821790 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:41.822652 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:41.822875 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:41.822995 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:41.823303 (+ 308us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:26:41.823760 (+ 457us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:26:41.823889 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:41.823953 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 18:26:41.824006 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 18:26:41.829810 12274 raft_consensus.cc:1261] T 6ca94d8cee0e4c56b2a2888a81d2b291 P 5211a6d11adb4533a294f1306fbce8f0 [term 1 FOLLOWER]: Refusing update from remote peer ca8d18fa287d4ed9bd77936ccd335850: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:26:41.829810 12557 raft_consensus.cc:1261] T 6ca94d8cee0e4c56b2a2888a81d2b291 P f6ef84bdc5034338a990eb42dee4b2f0 [term 1 FOLLOWER]: Refusing update from remote peer ca8d18fa287d4ed9bd77936ccd335850: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:26:41.830538 12637 consensus_queue.cc:1035] T 6ca94d8cee0e4c56b2a2888a81d2b291 P ca8d18fa287d4ed9bd77936ccd335850 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6ef84bdc5034338a990eb42dee4b2f0" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 36333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:26:41.830789 12635 consensus_queue.cc:1035] T 6ca94d8cee0e4c56b2a2888a81d2b291 P ca8d18fa287d4ed9bd77936ccd335850 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5211a6d11adb4533a294f1306fbce8f0" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 38211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:26:41.840924 12642 mvcc.cc:205] Tried to move back new op lower bound from 7316330913087967232 to 7316330913015504896. Current Snapshot: MvccSnapshot[applied={T|T < 7316330913087967232}]
I20260808 18:26:41.848852 12117 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54818:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:26:41.849035 12117 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54818:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:26:41.851576 12117 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5ea358857ebb435b951751d828ba27c9: Sending DeleteTablet for 3 replicas of tablet 6ca94d8cee0e4c56b2a2888a81d2b291
I20260808 18:26:41.852406 12536 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ca94d8cee0e4c56b2a2888a81d2b291 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:26:41 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:46406
I20260808 18:26:41.852406 12251 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ca94d8cee0e4c56b2a2888a81d2b291 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:26:41 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:52902
I20260808 18:26:41.852471 12392 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ca94d8cee0e4c56b2a2888a81d2b291 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:26:41 UTC) from {username='kudu', principal='kudu/127.23.72.190@KRBTEST.COM'} at 127.0.0.1:36848
I20260808 18:26:41.852926 12652 tablet_replica.cc:330] T 6ca94d8cee0e4c56b2a2888a81d2b291 P f6ef84bdc5034338a990eb42dee4b2f0: stopping tablet replica
I20260808 18:26:41.852933 12653 tablet_replica.cc:330] T 6ca94d8cee0e4c56b2a2888a81d2b291 P 5211a6d11adb4533a294f1306fbce8f0: stopping tablet replica
I20260808 18:26:41.853248 12653 raft_consensus.cc:2229] T 6ca94d8cee0e4c56b2a2888a81d2b291 P 5211a6d11adb4533a294f1306fbce8f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:26:41.853248 12652 raft_consensus.cc:2229] T 6ca94d8cee0e4c56b2a2888a81d2b291 P f6ef84bdc5034338a990eb42dee4b2f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:26:41.853559 12652 raft_consensus.cc:2258] T 6ca94d8cee0e4c56b2a2888a81d2b291 P f6ef84bdc5034338a990eb42dee4b2f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:26:41.853559 12653 raft_consensus.cc:2258] T 6ca94d8cee0e4c56b2a2888a81d2b291 P 5211a6d11adb4533a294f1306fbce8f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:26:41.854061 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 12185
I20260808 18:26:41.857120 12651 tablet_replica.cc:330] T 6ca94d8cee0e4c56b2a2888a81d2b291 P ca8d18fa287d4ed9bd77936ccd335850: stopping tablet replica
I20260808 18:26:41.857399 12651 raft_consensus.cc:2229] T 6ca94d8cee0e4c56b2a2888a81d2b291 P ca8d18fa287d4ed9bd77936ccd335850 [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:26:41.857744 12652 ts_tablet_manager.cc:1905] T 6ca94d8cee0e4c56b2a2888a81d2b291 P f6ef84bdc5034338a990eb42dee4b2f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:26:41.857786 12651 raft_consensus.cc:2258] T 6ca94d8cee0e4c56b2a2888a81d2b291 P ca8d18fa287d4ed9bd77936ccd335850 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:26:41.861606 12652 ts_tablet_manager.cc:1918] T 6ca94d8cee0e4c56b2a2888a81d2b291 P f6ef84bdc5034338a990eb42dee4b2f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:26:41.861711 12652 log.cc:1196] T 6ca94d8cee0e4c56b2a2888a81d2b291 P f6ef84bdc5034338a990eb42dee4b2f0: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-2/wal/wals/6ca94d8cee0e4c56b2a2888a81d2b291
I20260808 18:26:41.861912 12651 ts_tablet_manager.cc:1905] T 6ca94d8cee0e4c56b2a2888a81d2b291 P ca8d18fa287d4ed9bd77936ccd335850: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:26:41.862046 12652 ts_tablet_manager.cc:1939] T 6ca94d8cee0e4c56b2a2888a81d2b291 P f6ef84bdc5034338a990eb42dee4b2f0: Deleting consensus metadata
I20260808 18:26:41.863451 12101 catalog_manager.cc:4841] TS f6ef84bdc5034338a990eb42dee4b2f0 (127.23.72.131:36333): tablet 6ca94d8cee0e4c56b2a2888a81d2b291 (table test-table [id=5bbd51b016324ae9bf93e780d1570c21]) successfully deleted
I20260808 18:26:41.864588 12651 ts_tablet_manager.cc:1918] T 6ca94d8cee0e4c56b2a2888a81d2b291 P ca8d18fa287d4ed9bd77936ccd335850: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:26:41.864687 12651 log.cc:1196] T 6ca94d8cee0e4c56b2a2888a81d2b291 P ca8d18fa287d4ed9bd77936ccd335850: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213345431840-23842-0/minicluster-data/ts-1/wal/wals/6ca94d8cee0e4c56b2a2888a81d2b291
I20260808 18:26:41.865039 12651 ts_tablet_manager.cc:1939] T 6ca94d8cee0e4c56b2a2888a81d2b291 P ca8d18fa287d4ed9bd77936ccd335850: Deleting consensus metadata
I20260808 18:26:41.866210 12103 catalog_manager.cc:4841] TS ca8d18fa287d4ed9bd77936ccd335850 (127.23.72.130:34585): tablet 6ca94d8cee0e4c56b2a2888a81d2b291 (table test-table [id=5bbd51b016324ae9bf93e780d1570c21]) successfully deleted
W20260808 18:26:41.867013 12104 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.72.129:38211 (error 108)
I20260808 18:26:41.867949 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 12328
I20260808 18:26:41.868173 12619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:41.867843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:38211 (local address 127.0.0.1:52906)
0808 18:26:41.867991 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:41.868073 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.129:38211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 18:26:41.870924 12104 catalog_manager.cc:4584] TS 5211a6d11adb4533a294f1306fbce8f0 (127.23.72.129:38211): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6ca94d8cee0e4c56b2a2888a81d2b291: Network error: Client connection negotiation failed: client connection to 127.23.72.129:38211: connect: Connection refused (error 111)
I20260808 18:26:41.874683 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 12466
I20260808 18:26:41.881673 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 12085
2026-08-08T18:26:41Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2329 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-08T18:26:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:26:41Z Disabled control of system clock
I20260808 18:26:41.914585 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:43531
--webserver_interface=127.23.72.190
--webserver_port=0
--builtin_ntp_servers=127.23.72.148:44395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:43531
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 18:26:42.025820 12658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:26:42.026078 12658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:26:42.026113 12658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:26:42.030220 12658 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:26:42.030268 12658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:26:42.030284 12658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:26:42.030300 12658 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:26:42.030314 12658 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:26:42.035003 12658 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:44395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:43531
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:43531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.72.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12658
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:26:42.036118 12658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:26:42.037472 12658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:26:42.045912 12664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:42.045872 12663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:42.045864 12666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:42.046110 12658 server_base.cc:1034] running on GCE node
I20260808 18:26:42.046747 12658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:26:42.047883 12658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:26:42.049079 12658 hybrid_clock.cc:648] HybridClock initialized: now 1786213602049067 us; error 26 us; skew 500 ppm
I20260808 18:26:42.050875 12658 webserver.cc:460] Webserver started at http://127.23.72.190:46385/ using document root <none> and password file <none>
I20260808 18:26:42.051540 12658 fs_manager.cc:359] Metadata directory not provided
I20260808 18:26:42.051631 12658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:26:42.051857 12658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:26:42.053815 12658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "c4e3f511d2514ed1934fb34ce3ae3c69"
format_stamp: "Formatted at 2026-08-08 18:26:42 on dist-test-slave-5k54"
server_key: "168aa8ca45f5adf59423dd972eaae8f2"
server_key_iv: "66614c48e5955e69ad3f6450e8a0165c"
server_key_version: "encryptionkey@0"
I20260808 18:26:42.054385 12658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "c4e3f511d2514ed1934fb34ce3ae3c69"
format_stamp: "Formatted at 2026-08-08 18:26:42 on dist-test-slave-5k54"
server_key: "168aa8ca45f5adf59423dd972eaae8f2"
server_key_iv: "66614c48e5955e69ad3f6450e8a0165c"
server_key_version: "encryptionkey@0"
I20260808 18:26:42.057873 12658 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 18:26:42.060226 12672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:42.061373 12658 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:26:42.061517 12658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "c4e3f511d2514ed1934fb34ce3ae3c69"
format_stamp: "Formatted at 2026-08-08 18:26:42 on dist-test-slave-5k54"
server_key: "168aa8ca45f5adf59423dd972eaae8f2"
server_key_iv: "66614c48e5955e69ad3f6450e8a0165c"
server_key_version: "encryptionkey@0"
I20260808 18:26:42.061625 12658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:26:42.073139 12658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:26:42.073745 12658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:26:42.073944 12658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:26:42.080746 12658 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.190:43531
I20260808 18:26:42.080754 12727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.190:43531 every 8 connection(s)
I20260808 18:26:42.081664 12658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:26:42.084681 12728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:26:42.089753 12728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4e3f511d2514ed1934fb34ce3ae3c69: Bootstrap starting.
I20260808 18:26:42.090456 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 12658
I20260808 18:26:42.090590 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:26:42.090859 23842 external_mini_cluster.cc:1428] Setting key 3ca082e06fdf87dfbe09f7bd0480c2d8
I20260808 18:26:42.092523 12728 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4e3f511d2514ed1934fb34ce3ae3c69: Neither blocks nor log segments found. Creating new log.
I20260808 18:26:42.093228 12728 log.cc:824] T 00000000000000000000000000000000 P c4e3f511d2514ed1934fb34ce3ae3c69: Log is configured to *not* fsync() on all Append() calls
I20260808 18:26:42.095306 12728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4e3f511d2514ed1934fb34ce3ae3c69: No bootstrap required, opened a new log
I20260808 18:26:42.099668 12728 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4e3f511d2514ed1934fb34ce3ae3c69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e3f511d2514ed1934fb34ce3ae3c69" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 43531 } }
I20260808 18:26:42.099949 12728 raft_consensus.cc:374] T 00000000000000000000000000000000 P c4e3f511d2514ed1934fb34ce3ae3c69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:26:42.100018 12728 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4e3f511d2514ed1934fb34ce3ae3c69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4e3f511d2514ed1934fb34ce3ae3c69, State: Initialized, Role: FOLLOWER
I20260808 18:26:42.100471 12728 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4e3f511d2514ed1934fb34ce3ae3c69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e3f511d2514ed1934fb34ce3ae3c69" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 43531 } }
I20260808 18:26:42.100574 12731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:42.091957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39394 (local address 127.23.72.190:43531)
0808 18:26:42.092541 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:42.092556 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:42.092589 (+ 33us) server_negotiation.cc:407] Connection header received
0808 18:26:42.093242 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:42.093258 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:42.093668 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:42.093979 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:42.094403 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:42.095833 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:42.096712 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:42.097428 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:42.097580 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:42.098401 (+ 821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:42.098408 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:42.098427 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:26:42.099005 (+ 578us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:42.099071 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:26:42.099099 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:42.099200 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:42.099350 (+ 150us) server_negotiation.cc:299] Negotiation successful
0808 18:26:42.099636 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":132,"threads_started":1}
I20260808 18:26:42.100627 12728 raft_consensus.cc:388] T 00000000000000000000000000000000 P c4e3f511d2514ed1934fb34ce3ae3c69 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:26:42.100682 12728 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4e3f511d2514ed1934fb34ce3ae3c69 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:26:42.100776 12728 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4e3f511d2514ed1934fb34ce3ae3c69 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:26:42.101653 12728 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4e3f511d2514ed1934fb34ce3ae3c69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e3f511d2514ed1934fb34ce3ae3c69" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 43531 } }
I20260808 18:26:42.102002 12728 leader_election.cc:302] T 00000000000000000000000000000000 P c4e3f511d2514ed1934fb34ce3ae3c69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4e3f511d2514ed1934fb34ce3ae3c69; no voters:
I20260808 18:26:42.102316 12728 leader_election.cc:288] T 00000000000000000000000000000000 P c4e3f511d2514ed1934fb34ce3ae3c69 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:26:42.102424 12733 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4e3f511d2514ed1934fb34ce3ae3c69 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:26:42.102667 12733 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4e3f511d2514ed1934fb34ce3ae3c69 [term 1 LEADER]: Becoming Leader. State: Replica: c4e3f511d2514ed1934fb34ce3ae3c69, State: Running, Role: LEADER
I20260808 18:26:42.103035 12733 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4e3f511d2514ed1934fb34ce3ae3c69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e3f511d2514ed1934fb34ce3ae3c69" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 43531 } }
I20260808 18:26:42.103588 12728 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4e3f511d2514ed1934fb34ce3ae3c69 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:26:42.104737 12734 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4e3f511d2514ed1934fb34ce3ae3c69 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4e3f511d2514ed1934fb34ce3ae3c69. Latest consensus state: current_term: 1 leader_uuid: "c4e3f511d2514ed1934fb34ce3ae3c69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e3f511d2514ed1934fb34ce3ae3c69" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 43531 } } }
I20260808 18:26:42.104879 12734 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4e3f511d2514ed1934fb34ce3ae3c69 [sys.catalog]: This master's current role is: LEADER
I20260808 18:26:42.105270 12745 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:26:42.105300 12735 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4e3f511d2514ed1934fb34ce3ae3c69 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c4e3f511d2514ed1934fb34ce3ae3c69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e3f511d2514ed1934fb34ce3ae3c69" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 43531 } } }
I20260808 18:26:42.105523 12735 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4e3f511d2514ed1934fb34ce3ae3c69 [sys.catalog]: This master's current role is: LEADER
I20260808 18:26:42.107849 12745 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:26:42.112764 12745 catalog_manager.cc:1347] Generated new cluster ID: 73b68e589edd4d349347408c157c4e57
I20260808 18:26:42.112828 12745 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:26:42.123333 12745 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:26:42.124197 12745 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:26:42.136142 12745 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c4e3f511d2514ed1934fb34ce3ae3c69: Generated new TSK 0
I20260808 18:26:42.136804 12745 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:26:42.141942 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.129:0
--local_ip_for_outbound_sockets=127.23.72.129
--webserver_interface=127.23.72.129
--webserver_port=0
--tserver_master_addrs=127.23.72.190:43531
--builtin_ntp_servers=127.23.72.148:44395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 18:26:42.256042 12756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:26:42.256291 12756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:26:42.256331 12756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:26:42.260550 12756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:26:42.260608 12756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:26:42.260689 12756 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.129
I20260808 18:26:42.265563 12756 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:44395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.72.129
--webserver_port=0
--tserver_master_addrs=127.23.72.190:43531
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.129
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:26:42.266649 12756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:26:42.268154 12756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:26:42.277856 12762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:42.277832 12763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:42.277834 12765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:42.278167 12756 server_base.cc:1034] running on GCE node
I20260808 18:26:42.278661 12756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:26:42.279368 12756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:26:42.280592 12756 hybrid_clock.cc:648] HybridClock initialized: now 1786213602280577 us; error 25 us; skew 500 ppm
I20260808 18:26:42.282558 12756 webserver.cc:460] Webserver started at http://127.23.72.129:40227/ using document root <none> and password file <none>
I20260808 18:26:42.283106 12756 fs_manager.cc:359] Metadata directory not provided
I20260808 18:26:42.283167 12756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:26:42.283349 12756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:26:42.285543 12756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-0/data/instance:
uuid: "8cb3781fec06423fbf8aa7fde8271d34"
format_stamp: "Formatted at 2026-08-08 18:26:42 on dist-test-slave-5k54"
server_key: "800e2fbb1a199b943d0b8464d63152d4"
server_key_iv: "61c8e98dbbd0b983e16cdc0c051c95fc"
server_key_version: "encryptionkey@0"
I20260808 18:26:42.286453 12756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance:
uuid: "8cb3781fec06423fbf8aa7fde8271d34"
format_stamp: "Formatted at 2026-08-08 18:26:42 on dist-test-slave-5k54"
server_key: "800e2fbb1a199b943d0b8464d63152d4"
server_key_iv: "61c8e98dbbd0b983e16cdc0c051c95fc"
server_key_version: "encryptionkey@0"
I20260808 18:26:42.290347 12756 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 18:26:42.292845 12772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:42.294098 12756 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:26:42.294239 12756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-0/wal
uuid: "8cb3781fec06423fbf8aa7fde8271d34"
format_stamp: "Formatted at 2026-08-08 18:26:42 on dist-test-slave-5k54"
server_key: "800e2fbb1a199b943d0b8464d63152d4"
server_key_iv: "61c8e98dbbd0b983e16cdc0c051c95fc"
server_key_version: "encryptionkey@0"
I20260808 18:26:42.294356 12756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:26:42.312315 12756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:26:42.312983 12756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:26:42.313180 12756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:26:42.313757 12756 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:26:42.314668 12756 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:26:42.314739 12756 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:42.314807 12756 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:26:42.314867 12756 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:42.324512 12756 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.129:35505
I20260808 18:26:42.324555 12898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.129:35505 every 8 connection(s)
I20260808 18:26:42.325455 12756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-0/data/info.pb
I20260808 18:26:42.329248 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 12756
I20260808 18:26:42.329356 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-0/wal/instance
I20260808 18:26:42.329674 23842 external_mini_cluster.cc:1428] Setting key aa2405913033b1be1721ae4efc1b78fe
I20260808 18:26:42.331840 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.130:0
--local_ip_for_outbound_sockets=127.23.72.130
--webserver_interface=127.23.72.130
--webserver_port=0
--tserver_master_addrs=127.23.72.190:43531
--builtin_ntp_servers=127.23.72.148:44395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 18:26:42.334954 12731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:42.327255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.129:44375 (local address 127.23.72.190:43531)
0808 18:26:42.327404 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:42.327408 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:42.328468 (+ 1060us) server_negotiation.cc:407] Connection header received
0808 18:26:42.330021 (+ 1553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:42.330025 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:42.330086 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:42.330163 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:42.331211 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:42.331829 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:42.333003 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:42.333231 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:42.333342 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:42.334450 (+ 1108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:42.334453 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:42.334456 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:26:42.334469 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:42.334491 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:26:42.334504 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:42.334580 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:42.334745 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 18:26:42.334807 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 18:26:42.335843 12901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:42.327673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:43531 (local address 127.23.72.129:44375)
0808 18:26:42.328307 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:42.328348 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:42.329801 (+ 1453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:42.330331 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:42.330338 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:42.330421 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:26:42.331063 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:42.331080 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:42.331955 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:42.331959 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:42.332862 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:42.332871 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:42.333381 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:42.333385 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:42.333556 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:42.334193 (+ 637us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:26:42.334195 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:26:42.334234 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:26:42.334295 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:26:42.334595 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:26:42.334606 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:26:42.334618 (+ 12us) client_negotiation.cc:769] Sending connection context
0808 18:26:42.334742 (+ 124us) client_negotiation.cc:240] Negotiation successful
0808 18:26:42.335012 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":124,"threads_started":1}
I20260808 18:26:42.337073 12899 heartbeater.cc:347] Connected to a master server at 127.23.72.190:43531
I20260808 18:26:42.337337 12899 heartbeater.cc:464] Registering TS with master...
I20260808 18:26:42.338038 12899 heartbeater.cc:511] Master 127.23.72.190:43531 requested a full tablet report, sending...
I20260808 18:26:42.339943 12690 ts_manager.cc:194] Registered new tserver with Master: 8cb3781fec06423fbf8aa7fde8271d34 (127.23.72.129:35505)
I20260808 18:26:42.341794 12690 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.72.129:44375
W20260808 18:26:42.445323 12902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:26:42.445568 12902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:26:42.445605 12902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:26:42.449612 12902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:26:42.449661 12902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:26:42.449713 12902 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.130
I20260808 18:26:42.454324 12902 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:44395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.72.130
--webserver_port=0
--tserver_master_addrs=127.23.72.190:43531
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.130
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:26:42.455281 12902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:26:42.456600 12902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:26:42.466135 12909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:42.466176 12907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:42.466167 12911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:42.466341 12902 server_base.cc:1034] running on GCE node
I20260808 18:26:42.466809 12902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:26:42.467473 12902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:26:42.468644 12902 hybrid_clock.cc:648] HybridClock initialized: now 1786213602468625 us; error 30 us; skew 500 ppm
I20260808 18:26:42.470402 12902 webserver.cc:460] Webserver started at http://127.23.72.130:39899/ using document root <none> and password file <none>
I20260808 18:26:42.471024 12902 fs_manager.cc:359] Metadata directory not provided
I20260808 18:26:42.471112 12902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:26:42.471315 12902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:26:42.473317 12902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-1/data/instance:
uuid: "4c6c2c52d6224a98aadd3f4186bd1025"
format_stamp: "Formatted at 2026-08-08 18:26:42 on dist-test-slave-5k54"
server_key: "aca254a0648b21041db24ac00fed19c6"
server_key_iv: "d9363319eb381d95fa77271b05b5113c"
server_key_version: "encryptionkey@0"
I20260808 18:26:42.473896 12902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance:
uuid: "4c6c2c52d6224a98aadd3f4186bd1025"
format_stamp: "Formatted at 2026-08-08 18:26:42 on dist-test-slave-5k54"
server_key: "aca254a0648b21041db24ac00fed19c6"
server_key_iv: "d9363319eb381d95fa77271b05b5113c"
server_key_version: "encryptionkey@0"
I20260808 18:26:42.477468 12902 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 18:26:42.479871 12917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:42.481053 12902 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:26:42.481177 12902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-1/wal
uuid: "4c6c2c52d6224a98aadd3f4186bd1025"
format_stamp: "Formatted at 2026-08-08 18:26:42 on dist-test-slave-5k54"
server_key: "aca254a0648b21041db24ac00fed19c6"
server_key_iv: "d9363319eb381d95fa77271b05b5113c"
server_key_version: "encryptionkey@0"
I20260808 18:26:42.481285 12902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:26:42.493010 12902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:26:42.493600 12902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:26:42.493794 12902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:26:42.494349 12902 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:26:42.495241 12902 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:26:42.495311 12902 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:42.495370 12902 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:26:42.495417 12902 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:42.504454 12902 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.130:39415
I20260808 18:26:42.504492 13036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.130:39415 every 8 connection(s)
I20260808 18:26:42.505446 12902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-1/data/info.pb
I20260808 18:26:42.508978 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 12902
I20260808 18:26:42.509130 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-1/wal/instance
I20260808 18:26:42.509440 23842 external_mini_cluster.cc:1428] Setting key 86887e8a4ea10b2e379860ea25c733ec
I20260808 18:26:42.511376 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.72.131:0
--local_ip_for_outbound_sockets=127.23.72.131
--webserver_interface=127.23.72.131
--webserver_port=0
--tserver_master_addrs=127.23.72.190:43531
--builtin_ntp_servers=127.23.72.148:44395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 18:26:42.515094 12731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:42.507175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.130:38073 (local address 127.23.72.190:43531)
0808 18:26:42.507329 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:42.507334 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:42.508467 (+ 1133us) server_negotiation.cc:407] Connection header received
0808 18:26:42.510142 (+ 1675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:42.510145 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:42.510203 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:42.510316 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:42.511206 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:42.511893 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:42.513052 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:42.513287 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:42.513397 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:42.514604 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:42.514607 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:42.514609 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:26:42.514621 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:42.514639 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:26:42.514647 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:42.514724 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:42.514890 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 18:26:42.514955 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 18:26:42.516076 13040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:42.507642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:43531 (local address 127.23.72.130:38073)
0808 18:26:42.508315 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:42.508354 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:42.509915 (+ 1561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:42.510483 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:42.510489 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:42.510574 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:26:42.511056 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:42.511068 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:42.512023 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:42.512027 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:42.512919 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:42.512928 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:42.513451 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:42.513455 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:42.513622 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:42.514318 (+ 696us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:26:42.514319 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:26:42.514359 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:26:42.514418 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:26:42.514738 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:26:42.514749 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:26:42.514756 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 18:26:42.514888 (+ 132us) client_negotiation.cc:240] Negotiation successful
0808 18:26:42.515195 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":115,"threads_started":1}
I20260808 18:26:42.517374 13038 heartbeater.cc:347] Connected to a master server at 127.23.72.190:43531
I20260808 18:26:42.517601 13038 heartbeater.cc:464] Registering TS with master...
I20260808 18:26:42.518108 13038 heartbeater.cc:511] Master 127.23.72.190:43531 requested a full tablet report, sending...
I20260808 18:26:42.519204 12690 ts_manager.cc:194] Registered new tserver with Master: 4c6c2c52d6224a98aadd3f4186bd1025 (127.23.72.130:39415)
I20260808 18:26:42.520090 12690 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.72.130:38073
W20260808 18:26:42.623409 13041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:26:42.623731 13041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:26:42.623770 13041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:26:42.627728 13041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:26:42.627774 13041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:26:42.627825 13041 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.72.131
I20260808 18:26:42.632413 13041 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:44395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.72.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.72.131
--webserver_port=0
--tserver_master_addrs=127.23.72.190:43531
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13041
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.72.131
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:26:42.633378 13041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:26:42.634666 13041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:26:42.644213 13047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:42.644215 13050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:42.644349 13041 server_base.cc:1034] running on GCE node
W20260808 18:26:42.644217 13048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:42.644857 13041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:26:42.645407 13041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:26:42.646569 13041 hybrid_clock.cc:648] HybridClock initialized: now 1786213602646550 us; error 31 us; skew 500 ppm
I20260808 18:26:42.648350 13041 webserver.cc:460] Webserver started at http://127.23.72.131:42111/ using document root <none> and password file <none>
I20260808 18:26:42.648922 13041 fs_manager.cc:359] Metadata directory not provided
I20260808 18:26:42.648993 13041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:26:42.649238 13041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:26:42.651229 13041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-2/data/instance:
uuid: "39ff4085eedd4d1bb173687613d4b54c"
format_stamp: "Formatted at 2026-08-08 18:26:42 on dist-test-slave-5k54"
server_key: "8d24f063904cde8fa1ae6fc418457297"
server_key_iv: "e33c812072bc2a70335be2ce27f1bb55"
server_key_version: "encryptionkey@0"
I20260808 18:26:42.651849 13041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance:
uuid: "39ff4085eedd4d1bb173687613d4b54c"
format_stamp: "Formatted at 2026-08-08 18:26:42 on dist-test-slave-5k54"
server_key: "8d24f063904cde8fa1ae6fc418457297"
server_key_iv: "e33c812072bc2a70335be2ce27f1bb55"
server_key_version: "encryptionkey@0"
I20260808 18:26:42.655277 13041 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:26:42.657594 13058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:42.658806 13041 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:26:42.658921 13041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-2/wal
uuid: "39ff4085eedd4d1bb173687613d4b54c"
format_stamp: "Formatted at 2026-08-08 18:26:42 on dist-test-slave-5k54"
server_key: "8d24f063904cde8fa1ae6fc418457297"
server_key_iv: "e33c812072bc2a70335be2ce27f1bb55"
server_key_version: "encryptionkey@0"
I20260808 18:26:42.659039 13041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:26:42.677311 13041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:26:42.677999 13041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:26:42.678197 13041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:26:42.678800 13041 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:26:42.679817 13041 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:26:42.679867 13041 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:42.679939 13041 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:26:42.679978 13041 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:42.689738 13041 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.131:44165
I20260808 18:26:42.689759 13183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.131:44165 every 8 connection(s)
I20260808 18:26:42.690618 13041 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-2/data/info.pb
I20260808 18:26:42.698623 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 13041
I20260808 18:26:42.698807 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-2/wal/instance
I20260808 18:26:42.699183 23842 external_mini_cluster.cc:1428] Setting key a70eda49ba66f4a58b8445ee326f58bd
I20260808 18:26:42.704552 12731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:42.692495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:52249 (local address 127.23.72.190:43531)
0808 18:26:42.692638 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:42.692641 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:42.693639 (+ 998us) server_negotiation.cc:407] Connection header received
0808 18:26:42.695069 (+ 1430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:42.695072 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:42.695132 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:42.695236 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:42.696182 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:42.696706 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:42.697984 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:42.698209 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:42.698318 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:42.703967 (+ 5649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:42.703971 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:42.703973 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:26:42.703984 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:42.704002 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:26:42.704010 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:42.704090 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:42.704343 (+ 253us) server_negotiation.cc:299] Negotiation successful
0808 18:26:42.704410 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 18:26:42.705487 13186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:42.692845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:43531 (local address 127.23.72.131:52249)
0808 18:26:42.693490 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:42.693530 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:42.694854 (+ 1324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:42.695411 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:42.695418 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:42.695556 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:26:42.696040 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:42.696053 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:42.696828 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:42.696832 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:42.697859 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:42.697868 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:42.698359 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:42.698364 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:42.698532 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:42.703682 (+ 5150us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:26:42.703684 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:26:42.703727 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:26:42.703793 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:26:42.704092 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:26:42.704105 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:26:42.704111 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 18:26:42.704314 (+ 203us) client_negotiation.cc:240] Negotiation successful
0808 18:26:42.704615 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":110,"threads_started":1}
I20260808 18:26:42.706524 13184 heartbeater.cc:347] Connected to a master server at 127.23.72.190:43531
I20260808 18:26:42.706763 13184 heartbeater.cc:464] Registering TS with master...
I20260808 18:26:42.707291 13184 heartbeater.cc:511] Master 127.23.72.190:43531 requested a full tablet report, sending...
I20260808 18:26:42.708302 12690 ts_manager.cc:194] Registered new tserver with Master: 39ff4085eedd4d1bb173687613d4b54c (127.23.72.131:44165)
I20260808 18:26:42.708875 12690 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.72.131:52249
I20260808 18:26:42.712982 23842 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:26:42.718441 12731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:42.714874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39408 (local address 127.23.72.190:43531)
0808 18:26:42.715042 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:42.715046 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:42.715128 (+ 82us) server_negotiation.cc:407] Connection header received
0808 18:26:42.715221 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:42.715223 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:42.715258 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:42.715321 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:42.715642 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:42.716143 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:42.716960 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:42.717163 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:42.717237 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:42.718023 (+ 786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:42.718025 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:42.718027 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:26:42.718037 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:42.718052 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:26:42.718059 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:42.718111 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:42.718212 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 18:26:42.718256 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260808 18:26:42.721099 12690 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39408:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:26:42.724506 12690 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:26:42.739085 13197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:42.732883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45358 (local address 127.23.72.131:44165)
0808 18:26:42.733301 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:42.733315 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:42.733734 (+ 419us) server_negotiation.cc:407] Connection header received
0808 18:26:42.734069 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:42.734075 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:42.734279 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:42.734430 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:26:42.735090 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:42.736332 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:42.737715 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:42.738375 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:42.738508 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:42.738705 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:42.738814 (+ 109us) server_negotiation.cc:299] Negotiation successful
0808 18:26:42.738906 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":106,"threads_started":1}
I20260808 18:26:42.739099 13198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:42.732771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.131:44165 (local address 127.0.0.1:45358)
0808 18:26:42.733591 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:42.733641 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:42.733845 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:42.734604 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:42.734610 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:42.734649 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:26:42.734942 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:42.734951 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:42.736468 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:42.736470 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:42.737592 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:42.737601 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:42.738512 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:42.738516 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:42.738621 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:42.738658 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 18:26:42.738776 (+ 118us) client_negotiation.cc:240] Negotiation successful
0808 18:26:42.738899 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643,"mutex_wait_us":120,"spinlock_wait_cycles":7168,"thread_start_us":90,"threads_started":1}
I20260808 18:26:42.739320 13199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:42.732876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.130:39415 (local address 127.0.0.1:40308)
0808 18:26:42.733747 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:42.733762 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:42.733859 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:42.734604 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:42.734610 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:42.734648 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:26:42.734942 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:42.734951 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:42.736441 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:42.736444 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:42.737639 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:42.737645 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:42.738934 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:42.738937 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:42.739024 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:42.739043 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 18:26:42.739106 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 18:26:42.739162 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":803,"mutex_wait_us":135,"spinlock_wait_cycles":7296,"thread_start_us":114,"threads_started":1}
I20260808 18:26:42.739511 13196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:42.733482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:35505 (local address 127.0.0.1:59244)
0808 18:26:42.734120 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:42.734136 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:42.734233 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:42.734963 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:42.734966 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:42.734983 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:26:42.735101 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:42.735108 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:42.736138 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:42.736143 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:42.737632 (+ 1489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:42.737644 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:42.738714 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:42.738717 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:42.738810 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:42.738833 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 18:26:42.738900 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 18:26:42.738973 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"mutex_wait_us":76,"thread_start_us":102,"threads_started":1}
I20260808 18:26:42.739820 13200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:42.733548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40308 (local address 127.23.72.130:39415)
0808 18:26:42.733989 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:42.734002 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:42.734027 (+ 25us) server_negotiation.cc:407] Connection header received
0808 18:26:42.734102 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:42.734106 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:42.734279 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:42.734430 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:26:42.735090 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:42.736314 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:42.737780 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:42.738572 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:42.738688 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:42.738873 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:42.739540 (+ 667us) server_negotiation.cc:299] Negotiation successful
0808 18:26:42.739651 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":140,"threads_started":1}
I20260808 18:26:42.740280 13201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:42.734042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59244 (local address 127.23.72.129:35505)
0808 18:26:42.734513 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:42.734532 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:42.734558 (+ 26us) server_negotiation.cc:407] Connection header received
0808 18:26:42.734637 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:42.734642 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:42.734824 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:42.734984 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:26:42.735224 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:42.736021 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:42.737781 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:42.738572 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:42.739774 (+ 1202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:42.739987 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:42.740039 (+ 52us) server_negotiation.cc:299] Negotiation successful
0808 18:26:42.740128 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":101,"threads_started":1}
I20260808 18:26:42.742388 13111 tablet_service.cc:1467] Processing CreateTablet for tablet b6b0c695450f405da7ea4f8ffdab39b3 (DEFAULT_TABLE table=test-table [id=801331f324be41758bf4b9c04d36b9fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:26:42.742583 12969 tablet_service.cc:1467] Processing CreateTablet for tablet b6b0c695450f405da7ea4f8ffdab39b3 (DEFAULT_TABLE table=test-table [id=801331f324be41758bf4b9c04d36b9fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:26:42.742583 12825 tablet_service.cc:1467] Processing CreateTablet for tablet b6b0c695450f405da7ea4f8ffdab39b3 (DEFAULT_TABLE table=test-table [id=801331f324be41758bf4b9c04d36b9fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:26:42.743412 13111 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6b0c695450f405da7ea4f8ffdab39b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:26:42.743636 12825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6b0c695450f405da7ea4f8ffdab39b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:26:42.743636 12969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6b0c695450f405da7ea4f8ffdab39b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:26:42.749567 13202 tablet_bootstrap.cc:492] T b6b0c695450f405da7ea4f8ffdab39b3 P 39ff4085eedd4d1bb173687613d4b54c: Bootstrap starting.
I20260808 18:26:42.750976 13204 tablet_bootstrap.cc:492] T b6b0c695450f405da7ea4f8ffdab39b3 P 4c6c2c52d6224a98aadd3f4186bd1025: Bootstrap starting.
I20260808 18:26:42.751744 13203 tablet_bootstrap.cc:492] T b6b0c695450f405da7ea4f8ffdab39b3 P 8cb3781fec06423fbf8aa7fde8271d34: Bootstrap starting.
I20260808 18:26:42.752079 13202 tablet_bootstrap.cc:654] T b6b0c695450f405da7ea4f8ffdab39b3 P 39ff4085eedd4d1bb173687613d4b54c: Neither blocks nor log segments found. Creating new log.
I20260808 18:26:42.753113 13202 log.cc:824] T b6b0c695450f405da7ea4f8ffdab39b3 P 39ff4085eedd4d1bb173687613d4b54c: Log is configured to *not* fsync() on all Append() calls
I20260808 18:26:42.753153 13204 tablet_bootstrap.cc:654] T b6b0c695450f405da7ea4f8ffdab39b3 P 4c6c2c52d6224a98aadd3f4186bd1025: Neither blocks nor log segments found. Creating new log.
I20260808 18:26:42.753898 13204 log.cc:824] T b6b0c695450f405da7ea4f8ffdab39b3 P 4c6c2c52d6224a98aadd3f4186bd1025: Log is configured to *not* fsync() on all Append() calls
I20260808 18:26:42.753961 13203 tablet_bootstrap.cc:654] T b6b0c695450f405da7ea4f8ffdab39b3 P 8cb3781fec06423fbf8aa7fde8271d34: Neither blocks nor log segments found. Creating new log.
I20260808 18:26:42.754786 13203 log.cc:824] T b6b0c695450f405da7ea4f8ffdab39b3 P 8cb3781fec06423fbf8aa7fde8271d34: Log is configured to *not* fsync() on all Append() calls
I20260808 18:26:42.755234 13202 tablet_bootstrap.cc:492] T b6b0c695450f405da7ea4f8ffdab39b3 P 39ff4085eedd4d1bb173687613d4b54c: No bootstrap required, opened a new log
I20260808 18:26:42.755453 13202 ts_tablet_manager.cc:1397] T b6b0c695450f405da7ea4f8ffdab39b3 P 39ff4085eedd4d1bb173687613d4b54c: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 18:26:42.756469 13204 tablet_bootstrap.cc:492] T b6b0c695450f405da7ea4f8ffdab39b3 P 4c6c2c52d6224a98aadd3f4186bd1025: No bootstrap required, opened a new log
I20260808 18:26:42.756597 13203 tablet_bootstrap.cc:492] T b6b0c695450f405da7ea4f8ffdab39b3 P 8cb3781fec06423fbf8aa7fde8271d34: No bootstrap required, opened a new log
I20260808 18:26:42.756651 13204 ts_tablet_manager.cc:1397] T b6b0c695450f405da7ea4f8ffdab39b3 P 4c6c2c52d6224a98aadd3f4186bd1025: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 18:26:42.756775 13203 ts_tablet_manager.cc:1397] T b6b0c695450f405da7ea4f8ffdab39b3 P 8cb3781fec06423fbf8aa7fde8271d34: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 18:26:42.759856 13202 raft_consensus.cc:348] T b6b0c695450f405da7ea4f8ffdab39b3 P 39ff4085eedd4d1bb173687613d4b54c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39ff4085eedd4d1bb173687613d4b54c" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 44165 } } peers { permanent_uuid: "4c6c2c52d6224a98aadd3f4186bd1025" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 39415 } } peers { permanent_uuid: "8cb3781fec06423fbf8aa7fde8271d34" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 35505 } }
I20260808 18:26:42.760138 13202 raft_consensus.cc:374] T b6b0c695450f405da7ea4f8ffdab39b3 P 39ff4085eedd4d1bb173687613d4b54c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:26:42.760197 13202 raft_consensus.cc:729] T b6b0c695450f405da7ea4f8ffdab39b3 P 39ff4085eedd4d1bb173687613d4b54c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39ff4085eedd4d1bb173687613d4b54c, State: Initialized, Role: FOLLOWER
I20260808 18:26:42.760207 13203 raft_consensus.cc:348] T b6b0c695450f405da7ea4f8ffdab39b3 P 8cb3781fec06423fbf8aa7fde8271d34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39ff4085eedd4d1bb173687613d4b54c" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 44165 } } peers { permanent_uuid: "4c6c2c52d6224a98aadd3f4186bd1025" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 39415 } } peers { permanent_uuid: "8cb3781fec06423fbf8aa7fde8271d34" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 35505 } }
I20260808 18:26:42.760475 13203 raft_consensus.cc:374] T b6b0c695450f405da7ea4f8ffdab39b3 P 8cb3781fec06423fbf8aa7fde8271d34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:26:42.760375 13204 raft_consensus.cc:348] T b6b0c695450f405da7ea4f8ffdab39b3 P 4c6c2c52d6224a98aadd3f4186bd1025 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39ff4085eedd4d1bb173687613d4b54c" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 44165 } } peers { permanent_uuid: "4c6c2c52d6224a98aadd3f4186bd1025" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 39415 } } peers { permanent_uuid: "8cb3781fec06423fbf8aa7fde8271d34" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 35505 } }
I20260808 18:26:42.760560 13203 raft_consensus.cc:729] T b6b0c695450f405da7ea4f8ffdab39b3 P 8cb3781fec06423fbf8aa7fde8271d34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cb3781fec06423fbf8aa7fde8271d34, State: Initialized, Role: FOLLOWER
I20260808 18:26:42.760599 13204 raft_consensus.cc:374] T b6b0c695450f405da7ea4f8ffdab39b3 P 4c6c2c52d6224a98aadd3f4186bd1025 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:26:42.760651 13204 raft_consensus.cc:729] T b6b0c695450f405da7ea4f8ffdab39b3 P 4c6c2c52d6224a98aadd3f4186bd1025 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c6c2c52d6224a98aadd3f4186bd1025, State: Initialized, Role: FOLLOWER
I20260808 18:26:42.760632 13202 consensus_queue.cc:260] T b6b0c695450f405da7ea4f8ffdab39b3 P 39ff4085eedd4d1bb173687613d4b54c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39ff4085eedd4d1bb173687613d4b54c" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 44165 } } peers { permanent_uuid: "4c6c2c52d6224a98aadd3f4186bd1025" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 39415 } } peers { permanent_uuid: "8cb3781fec06423fbf8aa7fde8271d34" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 35505 } }
I20260808 18:26:42.761030 13203 consensus_queue.cc:260] T b6b0c695450f405da7ea4f8ffdab39b3 P 8cb3781fec06423fbf8aa7fde8271d34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39ff4085eedd4d1bb173687613d4b54c" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 44165 } } peers { permanent_uuid: "4c6c2c52d6224a98aadd3f4186bd1025" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 39415 } } peers { permanent_uuid: "8cb3781fec06423fbf8aa7fde8271d34" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 35505 } }
I20260808 18:26:42.761096 13204 consensus_queue.cc:260] T b6b0c695450f405da7ea4f8ffdab39b3 P 4c6c2c52d6224a98aadd3f4186bd1025 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39ff4085eedd4d1bb173687613d4b54c" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 44165 } } peers { permanent_uuid: "4c6c2c52d6224a98aadd3f4186bd1025" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 39415 } } peers { permanent_uuid: "8cb3781fec06423fbf8aa7fde8271d34" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 35505 } }
I20260808 18:26:42.761438 13184 heartbeater.cc:503] Master 127.23.72.190:43531 was elected leader, sending a full tablet report...
I20260808 18:26:42.761643 13202 ts_tablet_manager.cc:1428] T b6b0c695450f405da7ea4f8ffdab39b3 P 39ff4085eedd4d1bb173687613d4b54c: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 18:26:42.761981 13203 ts_tablet_manager.cc:1428] T b6b0c695450f405da7ea4f8ffdab39b3 P 8cb3781fec06423fbf8aa7fde8271d34: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 18:26:42.761993 13204 ts_tablet_manager.cc:1428] T b6b0c695450f405da7ea4f8ffdab39b3 P 4c6c2c52d6224a98aadd3f4186bd1025: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 18:26:42.762308 13038 heartbeater.cc:503] Master 127.23.72.190:43531 was elected leader, sending a full tablet report...
I20260808 18:26:42.763391 12899 heartbeater.cc:503] Master 127.23.72.190:43531 was elected leader, sending a full tablet report...
W20260808 18:26:42.826681 12900 tablet.cc:2369] T b6b0c695450f405da7ea4f8ffdab39b3 P 8cb3781fec06423fbf8aa7fde8271d34: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:26:42.941092 13209 raft_consensus.cc:482] T b6b0c695450f405da7ea4f8ffdab39b3 P 39ff4085eedd4d1bb173687613d4b54c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:26:42.941385 13209 raft_consensus.cc:504] T b6b0c695450f405da7ea4f8ffdab39b3 P 39ff4085eedd4d1bb173687613d4b54c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39ff4085eedd4d1bb173687613d4b54c" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 44165 } } peers { permanent_uuid: "4c6c2c52d6224a98aadd3f4186bd1025" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 39415 } } peers { permanent_uuid: "8cb3781fec06423fbf8aa7fde8271d34" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 35505 } }
W20260808 18:26:42.941588 13185 tablet.cc:2369] T b6b0c695450f405da7ea4f8ffdab39b3 P 39ff4085eedd4d1bb173687613d4b54c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:26:42.942759 13209 leader_election.cc:288] T b6b0c695450f405da7ea4f8ffdab39b3 P 39ff4085eedd4d1bb173687613d4b54c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c6c2c52d6224a98aadd3f4186bd1025 (127.23.72.130:39415), 8cb3781fec06423fbf8aa7fde8271d34 (127.23.72.129:35505)
I20260808 18:26:42.946818 13186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:42.943072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:35505 (local address 127.23.72.131:49119)
0808 18:26:42.943217 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:42.943235 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:42.943387 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:42.943700 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:42.943703 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:42.943730 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:26:42.943854 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:42.943864 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:42.944793 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:42.944797 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:42.945685 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:42.945694 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:42.946375 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:42.946381 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:42.946496 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:42.946519 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 18:26:42.946591 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 18:26:42.946656 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":1}
I20260808 18:26:42.946851 13201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:42.943284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:49119 (local address 127.23.72.129:35505)
0808 18:26:42.943478 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:42.943483 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:42.943499 (+ 16us) server_negotiation.cc:407] Connection header received
0808 18:26:42.943558 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:42.943560 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:42.943601 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:42.943689 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:26:42.944009 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:42.944670 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:42.945815 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:42.946253 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:42.946349 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:42.946430 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:42.946624 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 18:26:42.946713 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 18:26:42.946940 13200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:42.943172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.72.131:52963 (local address 127.23.72.130:39415)
0808 18:26:42.943321 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:42.943326 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:42.943836 (+ 510us) server_negotiation.cc:407] Connection header received
0808 18:26:42.943940 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:42.943945 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:42.944004 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:42.944063 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:26:42.944350 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:42.944914 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:42.945937 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:42.946328 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:42.946465 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:42.946511 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:42.946729 (+ 218us) server_negotiation.cc:299] Negotiation successful
0808 18:26:42.946801 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 18:26:42.946954 13212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:42.943072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.130:39415 (local address 127.23.72.131:52963)
0808 18:26:42.943754 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:42.943766 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:26:42.943859 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:26:42.944093 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:26:42.944096 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:26:42.944113 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:26:42.944241 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:42.944248 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:42.945023 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:42.945026 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:42.945815 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:26:42.945824 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:26:42.946471 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:26:42.946474 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:26:42.946597 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:42.946621 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 18:26:42.946697 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 18:26:42.946760 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"mutex_wait_us":266,"thread_start_us":192,"threads_started":1}
I20260808 18:26:42.947561 12989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6b0c695450f405da7ea4f8ffdab39b3" candidate_uuid: "39ff4085eedd4d1bb173687613d4b54c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c6c2c52d6224a98aadd3f4186bd1025" is_pre_election: true
I20260808 18:26:42.947558 12847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6b0c695450f405da7ea4f8ffdab39b3" candidate_uuid: "39ff4085eedd4d1bb173687613d4b54c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cb3781fec06423fbf8aa7fde8271d34" is_pre_election: true
I20260808 18:26:42.947885 12989 raft_consensus.cc:2454] T b6b0c695450f405da7ea4f8ffdab39b3 P 4c6c2c52d6224a98aadd3f4186bd1025 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39ff4085eedd4d1bb173687613d4b54c in term 0.
I20260808 18:26:42.947885 12847 raft_consensus.cc:2454] T b6b0c695450f405da7ea4f8ffdab39b3 P 8cb3781fec06423fbf8aa7fde8271d34 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39ff4085eedd4d1bb173687613d4b54c in term 0.
I20260808 18:26:42.948395 13062 leader_election.cc:302] T b6b0c695450f405da7ea4f8ffdab39b3 P 39ff4085eedd4d1bb173687613d4b54c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39ff4085eedd4d1bb173687613d4b54c, 8cb3781fec06423fbf8aa7fde8271d34; no voters:
I20260808 18:26:42.948637 13209 raft_consensus.cc:2781] T b6b0c695450f405da7ea4f8ffdab39b3 P 39ff4085eedd4d1bb173687613d4b54c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:26:42.948729 13209 raft_consensus.cc:482] T b6b0c695450f405da7ea4f8ffdab39b3 P 39ff4085eedd4d1bb173687613d4b54c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:26:42.948781 13209 raft_consensus.cc:3037] T b6b0c695450f405da7ea4f8ffdab39b3 P 39ff4085eedd4d1bb173687613d4b54c [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:26:42.949826 13209 raft_consensus.cc:504] T b6b0c695450f405da7ea4f8ffdab39b3 P 39ff4085eedd4d1bb173687613d4b54c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39ff4085eedd4d1bb173687613d4b54c" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 44165 } } peers { permanent_uuid: "4c6c2c52d6224a98aadd3f4186bd1025" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 39415 } } peers { permanent_uuid: "8cb3781fec06423fbf8aa7fde8271d34" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 35505 } }
I20260808 18:26:42.950183 13209 leader_election.cc:288] T b6b0c695450f405da7ea4f8ffdab39b3 P 39ff4085eedd4d1bb173687613d4b54c [CANDIDATE]: Term 1 election: Requested vote from peers 4c6c2c52d6224a98aadd3f4186bd1025 (127.23.72.130:39415), 8cb3781fec06423fbf8aa7fde8271d34 (127.23.72.129:35505)
I20260808 18:26:42.950583 12847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6b0c695450f405da7ea4f8ffdab39b3" candidate_uuid: "39ff4085eedd4d1bb173687613d4b54c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cb3781fec06423fbf8aa7fde8271d34"
I20260808 18:26:42.950575 12989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6b0c695450f405da7ea4f8ffdab39b3" candidate_uuid: "39ff4085eedd4d1bb173687613d4b54c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c6c2c52d6224a98aadd3f4186bd1025"
I20260808 18:26:42.950706 12847 raft_consensus.cc:3037] T b6b0c695450f405da7ea4f8ffdab39b3 P 8cb3781fec06423fbf8aa7fde8271d34 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:26:42.950707 12989 raft_consensus.cc:3037] T b6b0c695450f405da7ea4f8ffdab39b3 P 4c6c2c52d6224a98aadd3f4186bd1025 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:26:42.952044 12989 raft_consensus.cc:2454] T b6b0c695450f405da7ea4f8ffdab39b3 P 4c6c2c52d6224a98aadd3f4186bd1025 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39ff4085eedd4d1bb173687613d4b54c in term 1.
I20260808 18:26:42.952044 12847 raft_consensus.cc:2454] T b6b0c695450f405da7ea4f8ffdab39b3 P 8cb3781fec06423fbf8aa7fde8271d34 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39ff4085eedd4d1bb173687613d4b54c in term 1.
I20260808 18:26:42.952370 13062 leader_election.cc:302] T b6b0c695450f405da7ea4f8ffdab39b3 P 39ff4085eedd4d1bb173687613d4b54c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39ff4085eedd4d1bb173687613d4b54c, 8cb3781fec06423fbf8aa7fde8271d34; no voters:
I20260808 18:26:42.952636 13209 raft_consensus.cc:2781] T b6b0c695450f405da7ea4f8ffdab39b3 P 39ff4085eedd4d1bb173687613d4b54c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:26:42.952827 13209 raft_consensus.cc:686] T b6b0c695450f405da7ea4f8ffdab39b3 P 39ff4085eedd4d1bb173687613d4b54c [term 1 LEADER]: Becoming Leader. State: Replica: 39ff4085eedd4d1bb173687613d4b54c, State: Running, Role: LEADER
I20260808 18:26:42.953152 13209 consensus_queue.cc:237] T b6b0c695450f405da7ea4f8ffdab39b3 P 39ff4085eedd4d1bb173687613d4b54c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39ff4085eedd4d1bb173687613d4b54c" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 44165 } } peers { permanent_uuid: "4c6c2c52d6224a98aadd3f4186bd1025" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 39415 } } peers { permanent_uuid: "8cb3781fec06423fbf8aa7fde8271d34" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 35505 } }
I20260808 18:26:42.956487 12687 catalog_manager.cc:5495] T b6b0c695450f405da7ea4f8ffdab39b3 P 39ff4085eedd4d1bb173687613d4b54c reported cstate change: term changed from 0 to 1, leader changed from <none> to 39ff4085eedd4d1bb173687613d4b54c (127.23.72.131). New cstate: current_term: 1 leader_uuid: "39ff4085eedd4d1bb173687613d4b54c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39ff4085eedd4d1bb173687613d4b54c" member_type: VOTER last_known_addr { host: "127.23.72.131" port: 44165 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c6c2c52d6224a98aadd3f4186bd1025" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 39415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8cb3781fec06423fbf8aa7fde8271d34" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 35505 } health_report { overall_health: UNKNOWN } } }
I20260808 18:26:42.976958 13197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:42.973289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45370 (local address 127.23.72.131:44165)
0808 18:26:42.973447 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:42.973452 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:42.973466 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:26:42.973640 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:42.973642 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:42.973696 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:42.973826 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:26:42.974077 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:42.974582 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:42.975467 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:42.975686 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:42.975795 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:42.976113 (+ 318us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:26:42.976534 (+ 421us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:26:42.976669 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:42.976730 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 18:26:42.976783 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 18:26:42.983019 12847 raft_consensus.cc:1261] T b6b0c695450f405da7ea4f8ffdab39b3 P 8cb3781fec06423fbf8aa7fde8271d34 [term 1 FOLLOWER]: Refusing update from remote peer 39ff4085eedd4d1bb173687613d4b54c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:26:42.983019 12989 raft_consensus.cc:1261] T b6b0c695450f405da7ea4f8ffdab39b3 P 4c6c2c52d6224a98aadd3f4186bd1025 [term 1 FOLLOWER]: Refusing update from remote peer 39ff4085eedd4d1bb173687613d4b54c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:26:42.983861 13209 consensus_queue.cc:1035] T b6b0c695450f405da7ea4f8ffdab39b3 P 39ff4085eedd4d1bb173687613d4b54c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c6c2c52d6224a98aadd3f4186bd1025" member_type: VOTER last_known_addr { host: "127.23.72.130" port: 39415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:26:42.984077 13213 consensus_queue.cc:1035] T b6b0c695450f405da7ea4f8ffdab39b3 P 39ff4085eedd4d1bb173687613d4b54c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8cb3781fec06423fbf8aa7fde8271d34" member_type: VOTER last_known_addr { host: "127.23.72.129" port: 35505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:26:42.991915 13217 mvcc.cc:205] Tried to move back new op lower bound from 7316330917810638848 to 7316330917698723840. Current Snapshot: MvccSnapshot[applied={T|T < 7316330917810638848}]
I20260808 18:26:42.992338 13220 mvcc.cc:205] Tried to move back new op lower bound from 7316330917810638848 to 7316330917698723840. Current Snapshot: MvccSnapshot[applied={T|T < 7316330917810638848}]
I20260808 18:26:43.001518 12687 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39408:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:26:43.001698 12687 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39408:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:26:43.004235 12687 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c4e3f511d2514ed1934fb34ce3ae3c69: Sending DeleteTablet for 3 replicas of tablet b6b0c695450f405da7ea4f8ffdab39b3
I20260808 18:26:43.005098 13111 tablet_service.cc:1514] Processing DeleteTablet for tablet b6b0c695450f405da7ea4f8ffdab39b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:26:42 UTC) from {username='slave'} at 127.0.0.1:45358
I20260808 18:26:43.005211 12969 tablet_service.cc:1514] Processing DeleteTablet for tablet b6b0c695450f405da7ea4f8ffdab39b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:26:42 UTC) from {username='slave'} at 127.0.0.1:40308
I20260808 18:26:43.005357 12825 tablet_service.cc:1514] Processing DeleteTablet for tablet b6b0c695450f405da7ea4f8ffdab39b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:26:42 UTC) from {username='slave'} at 127.0.0.1:59244
I20260808 18:26:43.005779 13228 tablet_replica.cc:330] T b6b0c695450f405da7ea4f8ffdab39b3 P 4c6c2c52d6224a98aadd3f4186bd1025: stopping tablet replica
I20260808 18:26:43.005879 13227 tablet_replica.cc:330] T b6b0c695450f405da7ea4f8ffdab39b3 P 39ff4085eedd4d1bb173687613d4b54c: stopping tablet replica
I20260808 18:26:43.006191 13228 raft_consensus.cc:2229] T b6b0c695450f405da7ea4f8ffdab39b3 P 4c6c2c52d6224a98aadd3f4186bd1025 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:26:43.006191 13227 raft_consensus.cc:2229] T b6b0c695450f405da7ea4f8ffdab39b3 P 39ff4085eedd4d1bb173687613d4b54c [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:26:43.006479 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 12756
I20260808 18:26:43.007035 13227 raft_consensus.cc:2258] T b6b0c695450f405da7ea4f8ffdab39b3 P 39ff4085eedd4d1bb173687613d4b54c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:26:43.007167 13228 raft_consensus.cc:2258] T b6b0c695450f405da7ea4f8ffdab39b3 P 4c6c2c52d6224a98aadd3f4186bd1025 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:26:43.008854 13227 ts_tablet_manager.cc:1905] T b6b0c695450f405da7ea4f8ffdab39b3 P 39ff4085eedd4d1bb173687613d4b54c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:26:43.008850 13228 ts_tablet_manager.cc:1905] T b6b0c695450f405da7ea4f8ffdab39b3 P 4c6c2c52d6224a98aadd3f4186bd1025: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:26:43.011698 13227 ts_tablet_manager.cc:1918] T b6b0c695450f405da7ea4f8ffdab39b3 P 39ff4085eedd4d1bb173687613d4b54c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:26:43.011698 13228 ts_tablet_manager.cc:1918] T b6b0c695450f405da7ea4f8ffdab39b3 P 4c6c2c52d6224a98aadd3f4186bd1025: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:26:43.011802 13227 log.cc:1196] T b6b0c695450f405da7ea4f8ffdab39b3 P 39ff4085eedd4d1bb173687613d4b54c: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-2/wal/wals/b6b0c695450f405da7ea4f8ffdab39b3
I20260808 18:26:43.011932 13228 log.cc:1196] T b6b0c695450f405da7ea4f8ffdab39b3 P 4c6c2c52d6224a98aadd3f4186bd1025: Deleting WAL directory at /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213345431840-23842-0/minicluster-data/ts-1/wal/wals/b6b0c695450f405da7ea4f8ffdab39b3
I20260808 18:26:43.012166 13227 ts_tablet_manager.cc:1939] T b6b0c695450f405da7ea4f8ffdab39b3 P 39ff4085eedd4d1bb173687613d4b54c: Deleting consensus metadata
I20260808 18:26:43.012419 13228 ts_tablet_manager.cc:1939] T b6b0c695450f405da7ea4f8ffdab39b3 P 4c6c2c52d6224a98aadd3f4186bd1025: Deleting consensus metadata
I20260808 18:26:43.013465 12673 catalog_manager.cc:4841] TS 39ff4085eedd4d1bb173687613d4b54c (127.23.72.131:44165): tablet b6b0c695450f405da7ea4f8ffdab39b3 (table test-table [id=801331f324be41758bf4b9c04d36b9fd]) successfully deleted
I20260808 18:26:43.013465 12674 catalog_manager.cc:4841] TS 4c6c2c52d6224a98aadd3f4186bd1025 (127.23.72.130:39415): tablet b6b0c695450f405da7ea4f8ffdab39b3 (table test-table [id=801331f324be41758bf4b9c04d36b9fd]) successfully deleted
W20260808 18:26:43.018250 12676 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.72.129:35505 (error 108)
I20260808 18:26:43.018345 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 12902
I20260808 18:26:43.020493 13196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:43.020133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.129:35505 (local address 127.0.0.1:59258)
0808 18:26:43.020322 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 18:26:43.020396 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.129:35505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
W20260808 18:26:43.020766 12676 catalog_manager.cc:4584] TS 8cb3781fec06423fbf8aa7fde8271d34 (127.23.72.129:35505): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b6b0c695450f405da7ea4f8ffdab39b3: Network error: Client connection negotiation failed: client connection to 127.23.72.129:35505: connect: Connection refused (error 111)
I20260808 18:26:43.026242 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 13041
I20260808 18:26:43.033176 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 12658
2026-08-08T18:26:43Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1150 ms)
[----------] 6 tests from EncryptionPolicyTest (10606 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213345431840-23842-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:26:43 dist-test-slave-5k54 krb5kdc[13233](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:26:43 dist-test-slave-5k54 krb5kdc[13233](info): set up 2 sockets
Aug 08 18:26:43 dist-test-slave-5k54 krb5kdc[13233](info): commencing operation
krb5kdc: starting...
W20260808 18:26:44.227746 23842 mini_kdc.cc:121] Time spent starting KDC: real 1.171s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Aug 08 18:26:44 dist-test-slave-5k54 krb5kdc[13233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:26:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:26:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213345431840-23842-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213345431840-23842-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 18:26:44.354066 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34971
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213345431840-23842-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34971
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:26:44.467548 13246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:26:44.467857 13246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:26:44.467939 13246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:26:44.472012 13246 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:26:44.472102 13246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:26:44.472141 13246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:26:44.472180 13246 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:26:44.472221 13246 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:26:44.477507 13246 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34971
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213345431840-23842-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:26:44.478605 13246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:26:44.479724 13246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:26:44.488526 13252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:44.488543 13254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:44.488615 13251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:44.488719 13246 server_base.cc:1034] running on GCE node
I20260808 18:26:44.489431 13246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:26:44.490540 13246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:26:44.491757 13246 hybrid_clock.cc:648] HybridClock initialized: now 1786213604491739 us; error 35 us; skew 500 ppm
Aug 08 18:26:44 dist-test-slave-5k54 krb5kdc[13233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213604, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:26:44.494544 13246 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 18:26:44.495524 13246 webserver.cc:460] Webserver started at http://127.0.0.1:37279/ using document root <none> and password file <none>
I20260808 18:26:44.496076 13246 fs_manager.cc:359] Metadata directory not provided
I20260808 18:26:44.496170 13246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:26:44.496409 13246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:26:44.498416 13246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "9bd9685d998141a6936d4065767e1196"
format_stamp: "Formatted at 2026-08-08 18:26:44 on dist-test-slave-5k54"
server_key: "d3f5d3942774a8e85e96dd26fc77aa8b"
server_key_iv: "7e532f314ec73ec439cf6191054af031"
server_key_version: "encryptionkey@0"
I20260808 18:26:44.498976 13246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "9bd9685d998141a6936d4065767e1196"
format_stamp: "Formatted at 2026-08-08 18:26:44 on dist-test-slave-5k54"
server_key: "d3f5d3942774a8e85e96dd26fc77aa8b"
server_key_iv: "7e532f314ec73ec439cf6191054af031"
server_key_version: "encryptionkey@0"
I20260808 18:26:44.502625 13246 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 18:26:44.505103 13261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:44.506317 13246 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:26:44.506484 13246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "9bd9685d998141a6936d4065767e1196"
format_stamp: "Formatted at 2026-08-08 18:26:44 on dist-test-slave-5k54"
server_key: "d3f5d3942774a8e85e96dd26fc77aa8b"
server_key_iv: "7e532f314ec73ec439cf6191054af031"
server_key_version: "encryptionkey@0"
I20260808 18:26:44.506598 13246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:26:44.532600 13246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:26:44.535624 13246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:26:44.535833 13246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:26:44.543207 13246 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34971
I20260808 18:26:44.543232 13316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34971 every 8 connection(s)
I20260808 18:26:44.544271 13246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:26:44.547015 13317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:26:44.551505 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 13246
I20260808 18:26:44.551664 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:26:44.552007 23842 external_mini_cluster.cc:1428] Setting key f9dff9be0d5e82c274bcf70cd65d80a1
I20260808 18:26:44.552953 13317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bd9685d998141a6936d4065767e1196: Bootstrap starting.
I20260808 18:26:44.555346 13317 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9bd9685d998141a6936d4065767e1196: Neither blocks nor log segments found. Creating new log.
I20260808 18:26:44.556080 13317 log.cc:824] T 00000000000000000000000000000000 P 9bd9685d998141a6936d4065767e1196: Log is configured to *not* fsync() on all Append() calls
I20260808 18:26:44.558020 13317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bd9685d998141a6936d4065767e1196: No bootstrap required, opened a new log
Aug 08 18:26:44 dist-test-slave-5k54 krb5kdc[13233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 18:26:44.561982 13317 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9bd9685d998141a6936d4065767e1196 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bd9685d998141a6936d4065767e1196" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34971 } }
I20260808 18:26:44.562219 13317 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9bd9685d998141a6936d4065767e1196 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:26:44.562317 13317 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9bd9685d998141a6936d4065767e1196 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bd9685d998141a6936d4065767e1196, State: Initialized, Role: FOLLOWER
I20260808 18:26:44.562789 13317 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9bd9685d998141a6936d4065767e1196 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bd9685d998141a6936d4065767e1196" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34971 } }
I20260808 18:26:44.562944 13317 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9bd9685d998141a6936d4065767e1196 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:26:44.563031 13317 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9bd9685d998141a6936d4065767e1196 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:26:44.563133 13317 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9bd9685d998141a6936d4065767e1196 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:26:44.564159 13317 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9bd9685d998141a6936d4065767e1196 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bd9685d998141a6936d4065767e1196" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34971 } }
I20260808 18:26:44.564518 13317 leader_election.cc:302] T 00000000000000000000000000000000 P 9bd9685d998141a6936d4065767e1196 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9bd9685d998141a6936d4065767e1196; no voters:
I20260808 18:26:44.564894 13317 leader_election.cc:288] T 00000000000000000000000000000000 P 9bd9685d998141a6936d4065767e1196 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:26:44.565021 13322 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9bd9685d998141a6936d4065767e1196 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:26:44.565222 13322 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9bd9685d998141a6936d4065767e1196 [term 1 LEADER]: Becoming Leader. State: Replica: 9bd9685d998141a6936d4065767e1196, State: Running, Role: LEADER
I20260808 18:26:44.565538 13322 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9bd9685d998141a6936d4065767e1196 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bd9685d998141a6936d4065767e1196" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34971 } }
I20260808 18:26:44.566269 13317 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9bd9685d998141a6936d4065767e1196 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:26:44.567309 13325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bd9685d998141a6936d4065767e1196 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9bd9685d998141a6936d4065767e1196. Latest consensus state: current_term: 1 leader_uuid: "9bd9685d998141a6936d4065767e1196" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bd9685d998141a6936d4065767e1196" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34971 } } }
I20260808 18:26:44.567456 13325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bd9685d998141a6936d4065767e1196 [sys.catalog]: This master's current role is: LEADER
I20260808 18:26:44.567803 13320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:44.553471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57773 (local address 127.0.0.1:34971)
0808 18:26:44.553898 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:44.553913 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:44.553948 (+ 35us) server_negotiation.cc:407] Connection header received
0808 18:26:44.554566 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:44.554581 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:44.554902 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:44.555259 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:44.556064 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:44.557109 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:44.557969 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:44.558484 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:44.558652 (+ 168us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:44.561607 (+ 2955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:44.561612 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:44.561624 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:26:44.561650 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:44.563744 (+ 2094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:44.564396 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:44.564404 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:44.564413 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:44.564519 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:44.564981 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:44.564985 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:44.564986 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:44.565520 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:44.565734 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:44.566441 (+ 707us) server_negotiation.cc:299] Negotiation successful
0808 18:26:44.566685 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":136,"threads_started":1}
I20260808 18:26:44.567853 13323 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bd9685d998141a6936d4065767e1196 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9bd9685d998141a6936d4065767e1196" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bd9685d998141a6936d4065767e1196" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34971 } } }
I20260808 18:26:44.568955 13323 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bd9685d998141a6936d4065767e1196 [sys.catalog]: This master's current role is: LEADER
I20260808 18:26:44.571134 23842 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260808 18:26:44.572424 13341 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9bd9685d998141a6936d4065767e1196: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:26:44.572505 13341 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:26:44.572571 13333 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:26:44.573469 13333 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:26:44.579154 13333 catalog_manager.cc:1347] Generated new cluster ID: 0053cf1bf2fe41f5a0e00152d0726885
I20260808 18:26:44.579214 13333 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:26:44.587359 13320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:44.573205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46885 (local address 127.0.0.1:34971)
0808 18:26:44.573315 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:44.573320 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:44.575582 (+ 2262us) server_negotiation.cc:407] Connection header received
0808 18:26:44.575703 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:44.575706 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:44.575751 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:44.575855 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:44.576664 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:44.577724 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:44.578921 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:44.579276 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:44.579350 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:44.581301 (+ 1951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:44.581305 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:44.581307 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:26:44.581332 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:44.585425 (+ 4093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:44.586055 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:44.586059 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:44.586061 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:44.586146 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:44.586474 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:44.586478 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:44.586480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:44.586682 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:44.586823 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:44.587108 (+ 285us) server_negotiation.cc:299] Negotiation successful
0808 18:26:44.587205 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 18:26:44.599375 13333 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:26:44.600268 13333 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:26:44.609527 13333 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9bd9685d998141a6936d4065767e1196: Generated new TSK 0
I20260808 18:26:44.610312 13333 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:26:44.616842 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 13246
W20260808 18:26:44.624719 13231 connection.cc:535] client connection to 127.0.0.1:34971 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34971): Connection reset by peer (error 104)
2026-08-08T18:26:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1583 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213345431840-23842-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:26:44 dist-test-slave-5k54 krb5kdc[13352](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:26:44 dist-test-slave-5k54 krb5kdc[13352](info): set up 2 sockets
Aug 08 18:26:44 dist-test-slave-5k54 krb5kdc[13352](info): commencing operation
krb5kdc: starting...
W20260808 18:26:45.702389 23842 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Aug 08 18:26:45 dist-test-slave-5k54 krb5kdc[13352](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:26:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:26:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213345431840-23842-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213345431840-23842-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 18:26:45.831511 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35203
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213345431840-23842-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35203
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:26:45.943948 13365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:26:45.944309 13365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:26:45.944408 13365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:26:45.948462 13365 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:26:45.948513 13365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:26:45.948532 13365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:26:45.948549 13365 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:26:45.948598 13365 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:26:45.953778 13365 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35203
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213345431840-23842-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:26:45.954780 13365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:26:45.955830 13365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:26:45.964040 13373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:45.964051 13371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:45.964051 13370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:45.964104 13365 server_base.cc:1034] running on GCE node
I20260808 18:26:45.964898 13365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:26:45.965975 13365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:26:45.967144 13365 hybrid_clock.cc:648] HybridClock initialized: now 1786213605967118 us; error 38 us; skew 500 ppm
Aug 08 18:26:45 dist-test-slave-5k54 krb5kdc[13352](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213605, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:26:45.969951 13365 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 18:26:45.970804 13365 webserver.cc:460] Webserver started at http://127.0.0.1:40525/ using document root <none> and password file <none>
I20260808 18:26:45.971335 13365 fs_manager.cc:359] Metadata directory not provided
I20260808 18:26:45.971421 13365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:26:45.971669 13365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:26:45.973698 13365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "349ca9777cdd466da3b1f8466d9ee63f"
format_stamp: "Formatted at 2026-08-08 18:26:45 on dist-test-slave-5k54"
server_key: "cf72561b3e3c0cc6f6b5971b4863b6db"
server_key_iv: "93894e80282aa462fca3fe707f9d60e3"
server_key_version: "encryptionkey@0"
I20260808 18:26:45.974246 13365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "349ca9777cdd466da3b1f8466d9ee63f"
format_stamp: "Formatted at 2026-08-08 18:26:45 on dist-test-slave-5k54"
server_key: "cf72561b3e3c0cc6f6b5971b4863b6db"
server_key_iv: "93894e80282aa462fca3fe707f9d60e3"
server_key_version: "encryptionkey@0"
I20260808 18:26:45.978021 13365 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 18:26:45.980481 13381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:45.981671 13365 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:26:45.981815 13365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "349ca9777cdd466da3b1f8466d9ee63f"
format_stamp: "Formatted at 2026-08-08 18:26:45 on dist-test-slave-5k54"
server_key: "cf72561b3e3c0cc6f6b5971b4863b6db"
server_key_iv: "93894e80282aa462fca3fe707f9d60e3"
server_key_version: "encryptionkey@0"
I20260808 18:26:45.981923 13365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:26:46.007242 13365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:26:46.010159 13365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:26:46.010378 13365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:26:46.017755 13365 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35203
I20260808 18:26:46.017783 13434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35203 every 8 connection(s)
I20260808 18:26:46.018700 13365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:26:46.021530 13435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:26:46.027037 13435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 349ca9777cdd466da3b1f8466d9ee63f: Bootstrap starting.
I20260808 18:26:46.028040 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 13365
I20260808 18:26:46.028200 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:26:46.028522 23842 external_mini_cluster.cc:1428] Setting key e5587c31141626ecdc9fbd3162499cf1
I20260808 18:26:46.029881 13435 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 349ca9777cdd466da3b1f8466d9ee63f: Neither blocks nor log segments found. Creating new log.
I20260808 18:26:46.030620 13435 log.cc:824] T 00000000000000000000000000000000 P 349ca9777cdd466da3b1f8466d9ee63f: Log is configured to *not* fsync() on all Append() calls
I20260808 18:26:46.032615 13435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 349ca9777cdd466da3b1f8466d9ee63f: No bootstrap required, opened a new log
I20260808 18:26:46.036305 13435 raft_consensus.cc:348] T 00000000000000000000000000000000 P 349ca9777cdd466da3b1f8466d9ee63f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "349ca9777cdd466da3b1f8466d9ee63f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35203 } }
I20260808 18:26:46.036556 13435 raft_consensus.cc:374] T 00000000000000000000000000000000 P 349ca9777cdd466da3b1f8466d9ee63f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:26:46.036633 13435 raft_consensus.cc:729] T 00000000000000000000000000000000 P 349ca9777cdd466da3b1f8466d9ee63f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 349ca9777cdd466da3b1f8466d9ee63f, State: Initialized, Role: FOLLOWER
I20260808 18:26:46.037083 13435 consensus_queue.cc:260] T 00000000000000000000000000000000 P 349ca9777cdd466da3b1f8466d9ee63f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "349ca9777cdd466da3b1f8466d9ee63f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35203 } }
I20260808 18:26:46.037238 13435 raft_consensus.cc:388] T 00000000000000000000000000000000 P 349ca9777cdd466da3b1f8466d9ee63f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:26:46.037329 13435 raft_consensus.cc:482] T 00000000000000000000000000000000 P 349ca9777cdd466da3b1f8466d9ee63f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:26:46.037465 13435 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 349ca9777cdd466da3b1f8466d9ee63f [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:26:46.037613 13438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:46.029669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49971 (local address 127.0.0.1:35203)
0808 18:26:46.030137 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:46.030153 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:46.030186 (+ 33us) server_negotiation.cc:407] Connection header received
0808 18:26:46.030879 (+ 693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:46.030897 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:46.031326 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:46.031717 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:46.032167 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:46.033177 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:46.034089 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:46.034569 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:46.034758 (+ 189us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:46.035762 (+ 1004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:46.035767 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:46.035780 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:26:46.036160 (+ 380us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:46.036213 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:26:46.036235 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:46.036377 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:46.036555 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 18:26:46.036691 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":132,"threads_started":1}
I20260808 18:26:46.038554 13435 raft_consensus.cc:504] T 00000000000000000000000000000000 P 349ca9777cdd466da3b1f8466d9ee63f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "349ca9777cdd466da3b1f8466d9ee63f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35203 } }
I20260808 18:26:46.038898 13435 leader_election.cc:302] T 00000000000000000000000000000000 P 349ca9777cdd466da3b1f8466d9ee63f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 349ca9777cdd466da3b1f8466d9ee63f; no voters:
I20260808 18:26:46.039222 13435 leader_election.cc:288] T 00000000000000000000000000000000 P 349ca9777cdd466da3b1f8466d9ee63f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:26:46.039285 13440 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 349ca9777cdd466da3b1f8466d9ee63f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:26:46.039531 13440 raft_consensus.cc:686] T 00000000000000000000000000000000 P 349ca9777cdd466da3b1f8466d9ee63f [term 1 LEADER]: Becoming Leader. State: Replica: 349ca9777cdd466da3b1f8466d9ee63f, State: Running, Role: LEADER
I20260808 18:26:46.039882 13440 consensus_queue.cc:237] T 00000000000000000000000000000000 P 349ca9777cdd466da3b1f8466d9ee63f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "349ca9777cdd466da3b1f8466d9ee63f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35203 } }
I20260808 18:26:46.040498 13435 sys_catalog.cc:564] T 00000000000000000000000000000000 P 349ca9777cdd466da3b1f8466d9ee63f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:26:46.040947 23842 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:26:46.043673 13442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 349ca9777cdd466da3b1f8466d9ee63f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 349ca9777cdd466da3b1f8466d9ee63f. Latest consensus state: current_term: 1 leader_uuid: "349ca9777cdd466da3b1f8466d9ee63f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "349ca9777cdd466da3b1f8466d9ee63f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35203 } } }
I20260808 18:26:46.043852 13442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 349ca9777cdd466da3b1f8466d9ee63f [sys.catalog]: This master's current role is: LEADER
I20260808 18:26:46.043717 13443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 349ca9777cdd466da3b1f8466d9ee63f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "349ca9777cdd466da3b1f8466d9ee63f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "349ca9777cdd466da3b1f8466d9ee63f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35203 } } }
I20260808 18:26:46.043990 13443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 349ca9777cdd466da3b1f8466d9ee63f [sys.catalog]: This master's current role is: LEADER
W20260808 18:26:46.046224 13462 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 349ca9777cdd466da3b1f8466d9ee63f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:26:46.046329 13462 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:26:46.046425 13464 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:26:46.047264 13464 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:26:46.047930 13438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:46.043410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45523 (local address 127.0.0.1:35203)
0808 18:26:46.043540 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:46.043544 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:46.043883 (+ 339us) server_negotiation.cc:407] Connection header received
0808 18:26:46.044040 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:46.044043 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:46.044085 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:46.044134 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:46.044535 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:46.045060 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:46.045941 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:46.046189 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:46.046296 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:46.047213 (+ 917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:46.047217 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:46.047220 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:26:46.047232 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:46.047266 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:26:46.047277 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:46.047380 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:46.047786 (+ 406us) server_negotiation.cc:299] Negotiation successful
0808 18:26:46.047836 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 18:26:46.053472 13464 catalog_manager.cc:1347] Generated new cluster ID: 2fa9e0ddea984dfaadbe65f7e4428c8a
I20260808 18:26:46.053531 13464 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:26:46.070655 13464 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:26:46.071693 13464 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:26:46.078334 13464 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 349ca9777cdd466da3b1f8466d9ee63f: Generated new TSK 0
I20260808 18:26:46.079018 13464 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:26:46.082768 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 13365
W20260808 18:26:46.090401 13350 connection.cc:535] client connection to 127.0.0.1:35203 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35203): Connection reset by peer (error 104)
2026-08-08T18:26:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1465 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213345431840-23842-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:26:46 dist-test-slave-5k54 krb5kdc[13472](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:26:46 dist-test-slave-5k54 krb5kdc[13472](info): set up 2 sockets
Aug 08 18:26:46 dist-test-slave-5k54 krb5kdc[13472](info): commencing operation
krb5kdc: starting...
W20260808 18:26:47.164618 23842 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Aug 08 18:26:47 dist-test-slave-5k54 krb5kdc[13472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:26:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:26:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213345431840-23842-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213345431840-23842-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 18:26:47.290570 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40319
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213345431840-23842-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40319
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:26:47.401317 13486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:26:47.401614 13486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:26:47.401686 13486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:26:47.405731 13486 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:26:47.405815 13486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:26:47.405855 13486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:26:47.405907 13486 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:26:47.405953 13486 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:26:47.411181 13486 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40319
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213345431840-23842-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13486
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:26:47.412235 13486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:26:47.413221 13486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 18:26:47.421540 13486 server_base.cc:1034] running on GCE node
W20260808 18:26:47.421456 13492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:47.421422 13494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:47.421430 13491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:47.422317 13486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:26:47.423395 13486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:26:47.424639 13486 hybrid_clock.cc:648] HybridClock initialized: now 1786213607424608 us; error 41 us; skew 500 ppm
Aug 08 18:26:47 dist-test-slave-5k54 krb5kdc[13472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213607, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:26:47.427623 13486 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 18:26:47.428537 13486 webserver.cc:460] Webserver started at http://127.0.0.1:36677/ using document root <none> and password file <none>
I20260808 18:26:47.429116 13486 fs_manager.cc:359] Metadata directory not provided
I20260808 18:26:47.429206 13486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:26:47.429445 13486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:26:47.431584 13486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "6db60b6c24454118b51d5fbe13a1091e"
format_stamp: "Formatted at 2026-08-08 18:26:47 on dist-test-slave-5k54"
server_key: "000a62b32f8ef90a127146bae15db4bb"
server_key_iv: "804137525fdb142661ff2e5dfd401a1d"
server_key_version: "encryptionkey@0"
I20260808 18:26:47.432196 13486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "6db60b6c24454118b51d5fbe13a1091e"
format_stamp: "Formatted at 2026-08-08 18:26:47 on dist-test-slave-5k54"
server_key: "000a62b32f8ef90a127146bae15db4bb"
server_key_iv: "804137525fdb142661ff2e5dfd401a1d"
server_key_version: "encryptionkey@0"
I20260808 18:26:47.436026 13486 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 18:26:47.438620 13501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:47.439989 13486 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:26:47.440146 13486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "6db60b6c24454118b51d5fbe13a1091e"
format_stamp: "Formatted at 2026-08-08 18:26:47 on dist-test-slave-5k54"
server_key: "000a62b32f8ef90a127146bae15db4bb"
server_key_iv: "804137525fdb142661ff2e5dfd401a1d"
server_key_version: "encryptionkey@0"
I20260808 18:26:47.440263 13486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:26:47.444777 13486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:26:47.447732 13486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:26:47.447929 13486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:26:47.454942 13486 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40319
I20260808 18:26:47.454973 13556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40319 every 8 connection(s)
I20260808 18:26:47.455897 13486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:26:47.457142 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 13486
I20260808 18:26:47.457304 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:26:47.457613 23842 external_mini_cluster.cc:1428] Setting key 2a20489905a4d320385b6c90cb779e91
I20260808 18:26:47.458812 13557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:26:47.463227 13560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:47.459096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55241 (local address 127.0.0.1:40319)
0808 18:26:47.459540 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:47.459560 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:47.459600 (+ 40us) server_negotiation.cc:407] Connection header received
0808 18:26:47.460442 (+ 842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:47.460464 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:47.460808 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:47.461166 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:47.461857 (+ 691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:47.461863 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:47.461877 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:26:47.461889 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:47.461956 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:26:47.461984 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:47.462123 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:47.462192 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 18:26:47.462341 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":127,"threads_started":1}
I20260808 18:26:47.465792 13557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6db60b6c24454118b51d5fbe13a1091e: Bootstrap starting.
I20260808 18:26:47.466095 23842 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:26:47.468161 13557 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6db60b6c24454118b51d5fbe13a1091e: Neither blocks nor log segments found. Creating new log.
I20260808 18:26:47.468991 13557 log.cc:824] T 00000000000000000000000000000000 P 6db60b6c24454118b51d5fbe13a1091e: Log is configured to *not* fsync() on all Append() calls
I20260808 18:26:47.469976 13560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:47.468107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45605 (local address 127.0.0.1:40319)
0808 18:26:47.468237 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:47.468244 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:47.468462 (+ 218us) server_negotiation.cc:407] Connection header received
0808 18:26:47.468531 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:47.468533 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:47.468567 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:47.468649 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:47.469525 (+ 876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:47.469529 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:47.469531 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:26:47.469544 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:47.469574 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:26:47.469593 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:47.469703 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:47.469741 (+ 38us) server_negotiation.cc:299] Negotiation successful
0808 18:26:47.469861 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 18:26:47.471299 13557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6db60b6c24454118b51d5fbe13a1091e: No bootstrap required, opened a new log
I20260808 18:26:47.474959 13557 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6db60b6c24454118b51d5fbe13a1091e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6db60b6c24454118b51d5fbe13a1091e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40319 } }
I20260808 18:26:47.475212 13557 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6db60b6c24454118b51d5fbe13a1091e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:26:47.475286 13557 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6db60b6c24454118b51d5fbe13a1091e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6db60b6c24454118b51d5fbe13a1091e, State: Initialized, Role: FOLLOWER
I20260808 18:26:47.475734 13557 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6db60b6c24454118b51d5fbe13a1091e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6db60b6c24454118b51d5fbe13a1091e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40319 } }
I20260808 18:26:47.475894 13557 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6db60b6c24454118b51d5fbe13a1091e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:26:47.475960 13557 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6db60b6c24454118b51d5fbe13a1091e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:26:47.476063 13557 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6db60b6c24454118b51d5fbe13a1091e [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:26:47.477089 13557 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6db60b6c24454118b51d5fbe13a1091e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6db60b6c24454118b51d5fbe13a1091e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40319 } }
I20260808 18:26:47.477417 13557 leader_election.cc:302] T 00000000000000000000000000000000 P 6db60b6c24454118b51d5fbe13a1091e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6db60b6c24454118b51d5fbe13a1091e; no voters:
I20260808 18:26:47.477716 13557 leader_election.cc:288] T 00000000000000000000000000000000 P 6db60b6c24454118b51d5fbe13a1091e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:26:47.477839 13567 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6db60b6c24454118b51d5fbe13a1091e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:26:47.478099 13567 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6db60b6c24454118b51d5fbe13a1091e [term 1 LEADER]: Becoming Leader. State: Replica: 6db60b6c24454118b51d5fbe13a1091e, State: Running, Role: LEADER
I20260808 18:26:47.478435 13567 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6db60b6c24454118b51d5fbe13a1091e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6db60b6c24454118b51d5fbe13a1091e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40319 } }
I20260808 18:26:47.478927 13557 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6db60b6c24454118b51d5fbe13a1091e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:26:47.480073 13569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6db60b6c24454118b51d5fbe13a1091e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6db60b6c24454118b51d5fbe13a1091e. Latest consensus state: current_term: 1 leader_uuid: "6db60b6c24454118b51d5fbe13a1091e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6db60b6c24454118b51d5fbe13a1091e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40319 } } }
I20260808 18:26:47.480197 13569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6db60b6c24454118b51d5fbe13a1091e [sys.catalog]: This master's current role is: LEADER
I20260808 18:26:47.480525 13568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6db60b6c24454118b51d5fbe13a1091e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6db60b6c24454118b51d5fbe13a1091e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6db60b6c24454118b51d5fbe13a1091e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40319 } } }
I20260808 18:26:47.480603 13576 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:26:47.481467 13568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6db60b6c24454118b51d5fbe13a1091e [sys.catalog]: This master's current role is: LEADER
I20260808 18:26:47.483769 13576 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:26:47.489557 13576 catalog_manager.cc:1347] Generated new cluster ID: 6d08f1e91bde4c0f8b7075950143e370
I20260808 18:26:47.489620 13576 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:26:47.507076 13576 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:26:47.507458 13576 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:26:47.515239 13576 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6db60b6c24454118b51d5fbe13a1091e: Generated new TSK 0
I20260808 18:26:47.516014 13576 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:26:47.523672 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 13486
2026-08-08T18:26:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1441 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213345431840-23842-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:26:47 dist-test-slave-5k54 krb5kdc[13588](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:26:47 dist-test-slave-5k54 krb5kdc[13588](info): set up 2 sockets
Aug 08 18:26:47 dist-test-slave-5k54 krb5kdc[13588](info): commencing operation
krb5kdc: starting...
W20260808 18:26:48.592370 23842 mini_kdc.cc:121] Time spent starting KDC: real 1.046s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Aug 08 18:26:48 dist-test-slave-5k54 krb5kdc[13588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:26:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:26:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213345431840-23842-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213345431840-23842-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 18:26:48.718920 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36523
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213345431840-23842-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36523
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:26:48.830528 13602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:26:48.830811 13602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:26:48.830884 13602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:26:48.835072 13602 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:26:48.835130 13602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:26:48.835150 13602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:26:48.835166 13602 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:26:48.835213 13602 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:26:48.840452 13602 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36523
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213345431840-23842-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13602
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:26:48.841539 13602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:26:48.842599 13602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:26:48.851058 13608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:48.851187 13602 server_base.cc:1034] running on GCE node
W20260808 18:26:48.851125 13611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:48.851058 13609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:48.851904 13602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:26:48.853067 13602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:26:48.854266 13602 hybrid_clock.cc:648] HybridClock initialized: now 1786213608854229 us; error 27 us; skew 500 ppm
Aug 08 18:26:48 dist-test-slave-5k54 krb5kdc[13588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213608, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:26:48.861846 13602 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 18:26:48.862850 13602 webserver.cc:460] Webserver started at http://127.0.0.1:40445/ using document root <none> and password file <none>
I20260808 18:26:48.863483 13602 fs_manager.cc:359] Metadata directory not provided
I20260808 18:26:48.863576 13602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:26:48.863811 13602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:26:48.865873 13602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "6b13e11022854562bc80eba1262c7c76"
format_stamp: "Formatted at 2026-08-08 18:26:48 on dist-test-slave-5k54"
server_key: "02e0d388b0b911741a70598d5c537b93"
server_key_iv: "714ba2f2f72a26b43aecc36581fce8a3"
server_key_version: "encryptionkey@0"
I20260808 18:26:48.866444 13602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "6b13e11022854562bc80eba1262c7c76"
format_stamp: "Formatted at 2026-08-08 18:26:48 on dist-test-slave-5k54"
server_key: "02e0d388b0b911741a70598d5c537b93"
server_key_iv: "714ba2f2f72a26b43aecc36581fce8a3"
server_key_version: "encryptionkey@0"
I20260808 18:26:48.870723 13602 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260808 18:26:48.873494 13618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:48.875029 13602 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 18:26:48.875174 13602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "6b13e11022854562bc80eba1262c7c76"
format_stamp: "Formatted at 2026-08-08 18:26:48 on dist-test-slave-5k54"
server_key: "02e0d388b0b911741a70598d5c537b93"
server_key_iv: "714ba2f2f72a26b43aecc36581fce8a3"
server_key_version: "encryptionkey@0"
I20260808 18:26:48.875321 13602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:26:48.889055 13602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:26:48.892287 13602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:26:48.892452 13602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:26:48.899731 13602 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36523
I20260808 18:26:48.899758 13676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36523 every 8 connection(s)
I20260808 18:26:48.900655 13602 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:26:48.903296 13677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:26:48.905463 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 13602
I20260808 18:26:48.905620 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:26:48.905962 23842 external_mini_cluster.cc:1428] Setting key 28caf9a29a933b5e305a73a7767951b9
I20260808 18:26:48.909374 13677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b13e11022854562bc80eba1262c7c76: Bootstrap starting.
I20260808 18:26:48.911571 13677 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b13e11022854562bc80eba1262c7c76: Neither blocks nor log segments found. Creating new log.
I20260808 18:26:48.912282 13677 log.cc:824] T 00000000000000000000000000000000 P 6b13e11022854562bc80eba1262c7c76: Log is configured to *not* fsync() on all Append() calls
I20260808 18:26:48.914551 13677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b13e11022854562bc80eba1262c7c76: No bootstrap required, opened a new log
Aug 08 18:26:48 dist-test-slave-5k54 krb5kdc[13588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 18:26:48.917423 13677 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b13e11022854562bc80eba1262c7c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b13e11022854562bc80eba1262c7c76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36523 } }
I20260808 18:26:48.917680 13677 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b13e11022854562bc80eba1262c7c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:26:48.917764 13677 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b13e11022854562bc80eba1262c7c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b13e11022854562bc80eba1262c7c76, State: Initialized, Role: FOLLOWER
I20260808 18:26:48.918220 13677 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b13e11022854562bc80eba1262c7c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b13e11022854562bc80eba1262c7c76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36523 } }
I20260808 18:26:48.918368 13677 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b13e11022854562bc80eba1262c7c76 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:26:48.918450 13677 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b13e11022854562bc80eba1262c7c76 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:26:48.918553 13677 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b13e11022854562bc80eba1262c7c76 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:26:48.919391 13677 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b13e11022854562bc80eba1262c7c76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b13e11022854562bc80eba1262c7c76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36523 } }
I20260808 18:26:48.919741 13677 leader_election.cc:302] T 00000000000000000000000000000000 P 6b13e11022854562bc80eba1262c7c76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b13e11022854562bc80eba1262c7c76; no voters:
I20260808 18:26:48.920115 13677 leader_election.cc:288] T 00000000000000000000000000000000 P 6b13e11022854562bc80eba1262c7c76 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:26:48.920187 13682 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b13e11022854562bc80eba1262c7c76 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:26:48.921159 13680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:48.907342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48455 (local address 127.0.0.1:36523)
0808 18:26:48.907785 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:48.907800 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:48.907840 (+ 40us) server_negotiation.cc:407] Connection header received
0808 18:26:48.908580 (+ 740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:48.908599 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:48.908984 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:48.909316 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:48.910149 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:48.911128 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:48.912100 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:48.912612 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:48.912832 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:48.915569 (+ 2737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:48.915574 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:48.915586 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:26:48.915613 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:48.918031 (+ 2418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:48.918477 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:48.918483 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:48.918493 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:48.918659 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:48.918942 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:48.918947 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:48.918948 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:48.919335 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:48.919570 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:48.919832 (+ 262us) server_negotiation.cc:299] Negotiation successful
0808 18:26:48.920160 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":120,"threads_started":1}
I20260808 18:26:48.921310 13677 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b13e11022854562bc80eba1262c7c76 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:26:48.920463 13682 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b13e11022854562bc80eba1262c7c76 [term 1 LEADER]: Becoming Leader. State: Replica: 6b13e11022854562bc80eba1262c7c76, State: Running, Role: LEADER
I20260808 18:26:48.922533 13682 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b13e11022854562bc80eba1262c7c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b13e11022854562bc80eba1262c7c76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36523 } }
I20260808 18:26:48.924153 23842 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:26:48.924274 13683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b13e11022854562bc80eba1262c7c76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b13e11022854562bc80eba1262c7c76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b13e11022854562bc80eba1262c7c76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36523 } } }
I20260808 18:26:48.924897 13683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b13e11022854562bc80eba1262c7c76 [sys.catalog]: This master's current role is: LEADER
I20260808 18:26:48.924597 13684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b13e11022854562bc80eba1262c7c76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b13e11022854562bc80eba1262c7c76. Latest consensus state: current_term: 1 leader_uuid: "6b13e11022854562bc80eba1262c7c76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b13e11022854562bc80eba1262c7c76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36523 } } }
I20260808 18:26:48.925177 13684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b13e11022854562bc80eba1262c7c76 [sys.catalog]: This master's current role is: LEADER
I20260808 18:26:48.925319 13698 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:26:48.928470 13698 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:26:48.934463 13698 catalog_manager.cc:1347] Generated new cluster ID: 88a3be3e2fa245eebb0f07d16586d2a0
I20260808 18:26:48.934580 13698 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:26:48.935550 13680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:48.926392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42235 (local address 127.0.0.1:36523)
0808 18:26:48.926508 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:48.926514 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:48.926702 (+ 188us) server_negotiation.cc:407] Connection header received
0808 18:26:48.926837 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:48.926839 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:48.926884 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:48.926961 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:48.927787 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:48.928506 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:48.929699 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:48.930048 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:48.930152 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:48.931611 (+ 1459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:48.931614 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:48.931617 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:26:48.931642 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:48.933595 (+ 1953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:48.934257 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:48.934261 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:48.934262 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:48.934362 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:48.934694 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:48.934697 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:48.934699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:48.934873 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:48.935011 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:48.935246 (+ 235us) server_negotiation.cc:299] Negotiation successful
0808 18:26:48.935375 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 18:26:48.948957 13698 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:26:48.950197 13698 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:26:48.956947 13698 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b13e11022854562bc80eba1262c7c76: Generated new TSK 0
I20260808 18:26:48.957769 13698 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:26:48.965536 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 13602
2026-08-08T18:26:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1441 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213345431840-23842-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:26:49 dist-test-slave-5k54 krb5kdc[13709](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:26:49 dist-test-slave-5k54 krb5kdc[13709](info): set up 2 sockets
Aug 08 18:26:49 dist-test-slave-5k54 krb5kdc[13709](info): commencing operation
krb5kdc: starting...
W20260808 18:26:50.111918 23842 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.008s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Aug 08 18:26:50 dist-test-slave-5k54 krb5kdc[13709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213610, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:26:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:26:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213345431840-23842-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213345431840-23842-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 18:26:50.237848 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42811
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213345431840-23842-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42811
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:26:50.347040 13724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:26:50.347297 13724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:26:50.347337 13724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:26:50.351209 13724 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:26:50.351264 13724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:26:50.351284 13724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:26:50.351300 13724 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:26:50.351315 13724 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:26:50.356532 13724 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42811
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213345431840-23842-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13724
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:26:50.357447 13724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:26:50.358433 13724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:26:50.366773 13730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:50.366817 13729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:50.366817 13732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:50.367177 13724 server_base.cc:1034] running on GCE node
I20260808 18:26:50.367730 13724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:26:50.368815 13724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:26:50.369987 13724 hybrid_clock.cc:648] HybridClock initialized: now 1786213610369962 us; error 37 us; skew 500 ppm
Aug 08 18:26:50 dist-test-slave-5k54 krb5kdc[13709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213610, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:26:50.372985 13724 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 18:26:50.373888 13724 webserver.cc:460] Webserver started at http://127.0.0.1:34023/ using document root <none> and password file <none>
I20260808 18:26:50.374433 13724 fs_manager.cc:359] Metadata directory not provided
I20260808 18:26:50.374518 13724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:26:50.374742 13724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:26:50.376793 13724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "81f74279594549e1a4ade3ff32c97f1d"
format_stamp: "Formatted at 2026-08-08 18:26:50 on dist-test-slave-5k54"
server_key: "40948aeb8e3a99ece5485cef47cce388"
server_key_iv: "8a448dd5ed245e2bbc733a7b854b1733"
server_key_version: "encryptionkey@0"
I20260808 18:26:50.377370 13724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "81f74279594549e1a4ade3ff32c97f1d"
format_stamp: "Formatted at 2026-08-08 18:26:50 on dist-test-slave-5k54"
server_key: "40948aeb8e3a99ece5485cef47cce388"
server_key_iv: "8a448dd5ed245e2bbc733a7b854b1733"
server_key_version: "encryptionkey@0"
I20260808 18:26:50.381131 13724 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 18:26:50.383505 13739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:50.384666 13724 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:26:50.384810 13724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "81f74279594549e1a4ade3ff32c97f1d"
format_stamp: "Formatted at 2026-08-08 18:26:50 on dist-test-slave-5k54"
server_key: "40948aeb8e3a99ece5485cef47cce388"
server_key_iv: "8a448dd5ed245e2bbc733a7b854b1733"
server_key_version: "encryptionkey@0"
I20260808 18:26:50.384953 13724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:26:50.417295 13724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:26:50.420557 13724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:26:50.420778 13724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:26:50.428193 13724 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42811
I20260808 18:26:50.428212 13795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42811 every 8 connection(s)
I20260808 18:26:50.429193 13724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:26:50.431957 13796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:26:50.434595 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 13724
I20260808 18:26:50.434759 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:26:50.435104 23842 external_mini_cluster.cc:1428] Setting key 6abea0c1a410b3c6cf6276c56de6c9a2
I20260808 18:26:50.437954 13796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81f74279594549e1a4ade3ff32c97f1d: Bootstrap starting.
I20260808 18:26:50.440575 13796 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81f74279594549e1a4ade3ff32c97f1d: Neither blocks nor log segments found. Creating new log.
I20260808 18:26:50.441365 13796 log.cc:824] T 00000000000000000000000000000000 P 81f74279594549e1a4ade3ff32c97f1d: Log is configured to *not* fsync() on all Append() calls
I20260808 18:26:50.443584 13796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81f74279594549e1a4ade3ff32c97f1d: No bootstrap required, opened a new log
I20260808 18:26:50.444183 13799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:50.436494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47437 (local address 127.0.0.1:42811)
0808 18:26:50.436923 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:50.436944 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:50.436992 (+ 48us) server_negotiation.cc:407] Connection header received
0808 18:26:50.437612 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:50.437628 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:50.437964 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:50.438303 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:50.438621 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:50.439648 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:50.440909 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:50.441369 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:50.441549 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:50.442480 (+ 931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:50.442484 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:50.442497 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:26:50.442875 (+ 378us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:50.442927 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:26:50.442949 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:50.443023 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:50.443187 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 18:26:50.443402 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":150,"threads_started":1}
I20260808 18:26:50.447027 23842 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:26:50.447659 13796 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81f74279594549e1a4ade3ff32c97f1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81f74279594549e1a4ade3ff32c97f1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42811 } }
I20260808 18:26:50.447937 13796 raft_consensus.cc:374] T 00000000000000000000000000000000 P 81f74279594549e1a4ade3ff32c97f1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:26:50.447999 13796 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81f74279594549e1a4ade3ff32c97f1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81f74279594549e1a4ade3ff32c97f1d, State: Initialized, Role: FOLLOWER
I20260808 18:26:50.448487 13796 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81f74279594549e1a4ade3ff32c97f1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81f74279594549e1a4ade3ff32c97f1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42811 } }
I20260808 18:26:50.448659 13796 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81f74279594549e1a4ade3ff32c97f1d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:26:50.448731 13796 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81f74279594549e1a4ade3ff32c97f1d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:26:50.448832 13796 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81f74279594549e1a4ade3ff32c97f1d [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:26:50.450004 13796 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81f74279594549e1a4ade3ff32c97f1d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81f74279594549e1a4ade3ff32c97f1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42811 } }
I20260808 18:26:50.450356 13796 leader_election.cc:302] T 00000000000000000000000000000000 P 81f74279594549e1a4ade3ff32c97f1d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81f74279594549e1a4ade3ff32c97f1d; no voters:
I20260808 18:26:50.450668 13796 leader_election.cc:288] T 00000000000000000000000000000000 P 81f74279594549e1a4ade3ff32c97f1d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:26:50.450779 13807 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81f74279594549e1a4ade3ff32c97f1d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:26:50.451056 13807 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81f74279594549e1a4ade3ff32c97f1d [term 1 LEADER]: Becoming Leader. State: Replica: 81f74279594549e1a4ade3ff32c97f1d, State: Running, Role: LEADER
I20260808 18:26:50.451406 13807 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81f74279594549e1a4ade3ff32c97f1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81f74279594549e1a4ade3ff32c97f1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42811 } }
I20260808 18:26:50.451907 13796 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81f74279594549e1a4ade3ff32c97f1d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:26:50.452603 13807 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81f74279594549e1a4ade3ff32c97f1d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81f74279594549e1a4ade3ff32c97f1d. Latest consensus state: current_term: 1 leader_uuid: "81f74279594549e1a4ade3ff32c97f1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81f74279594549e1a4ade3ff32c97f1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42811 } } }
I20260808 18:26:50.453118 13807 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81f74279594549e1a4ade3ff32c97f1d [sys.catalog]: This master's current role is: LEADER
I20260808 18:26:50.453091 13810 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81f74279594549e1a4ade3ff32c97f1d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "81f74279594549e1a4ade3ff32c97f1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81f74279594549e1a4ade3ff32c97f1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42811 } } }
I20260808 18:26:50.453492 13810 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81f74279594549e1a4ade3ff32c97f1d [sys.catalog]: This master's current role is: LEADER
I20260808 18:26:50.453578 13817 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:26:50.453927 13799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:50.449302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49681 (local address 127.0.0.1:42811)
0808 18:26:50.449420 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:50.449425 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:50.449598 (+ 173us) server_negotiation.cc:407] Connection header received
0808 18:26:50.449691 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:50.449694 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:50.449738 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:50.449828 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:50.450038 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:50.450778 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:50.451939 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:50.452259 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:50.452396 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:50.453201 (+ 805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:50.453206 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:50.453211 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:26:50.453225 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:50.453257 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:26:50.453272 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:50.453349 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:50.453749 (+ 400us) server_negotiation.cc:299] Negotiation successful
0808 18:26:50.453798 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":45}
I20260808 18:26:50.456972 13817 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:26:50.462302 13817 catalog_manager.cc:1347] Generated new cluster ID: f29db0180ac14b269695c69f091b0aef
I20260808 18:26:50.462371 13817 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:26:50.489933 13817 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:26:50.491158 13817 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:26:50.496567 13817 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 81f74279594549e1a4ade3ff32c97f1d: Generated new TSK 0
I20260808 18:26:50.497213 13817 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:26:50.502028 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 13724
2026-08-08T18:26:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1536 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213345431840-23842-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:26:50 dist-test-slave-5k54 krb5kdc[13829](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:26:50 dist-test-slave-5k54 krb5kdc[13829](info): set up 2 sockets
Aug 08 18:26:50 dist-test-slave-5k54 krb5kdc[13829](info): commencing operation
krb5kdc: starting...
W20260808 18:26:51.576776 23842 mini_kdc.cc:121] Time spent starting KDC: real 1.052s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Aug 08 18:26:51 dist-test-slave-5k54 krb5kdc[13829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:26:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:26:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213345431840-23842-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213345431840-23842-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 18:26:51.702891 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40657
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213345431840-23842-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40657
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:26:51.813736 13843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:26:51.814003 13843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:26:51.814047 13843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:26:51.818104 13843 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:26:51.818166 13843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:26:51.818187 13843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:26:51.818205 13843 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:26:51.818221 13843 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:26:51.823565 13843 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40657
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213345431840-23842-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13843
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:26:51.824612 13843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:26:51.825644 13843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:26:51.834470 13849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:51.834568 13843 server_base.cc:1034] running on GCE node
W20260808 18:26:51.834472 13851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:51.834470 13848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:51.835371 13843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:26:51.836462 13843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:26:51.837647 13843 hybrid_clock.cc:648] HybridClock initialized: now 1786213611837636 us; error 28 us; skew 500 ppm
Aug 08 18:26:51 dist-test-slave-5k54 krb5kdc[13829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213611, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:26:51.840591 13843 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 18:26:51.841487 13843 webserver.cc:460] Webserver started at http://127.0.0.1:40713/ using document root <none> and password file <none>
I20260808 18:26:51.842038 13843 fs_manager.cc:359] Metadata directory not provided
I20260808 18:26:51.842124 13843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:26:51.842353 13843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:26:51.844463 13843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "b622bd0cc4054d349e3e70bd979b0142"
format_stamp: "Formatted at 2026-08-08 18:26:51 on dist-test-slave-5k54"
server_key: "e407e3001609629cc35547854ad8207a"
server_key_iv: "ab30f37a7884cc4ced9525dea6ff8dda"
server_key_version: "encryptionkey@0"
I20260808 18:26:51.845044 13843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "b622bd0cc4054d349e3e70bd979b0142"
format_stamp: "Formatted at 2026-08-08 18:26:51 on dist-test-slave-5k54"
server_key: "e407e3001609629cc35547854ad8207a"
server_key_iv: "ab30f37a7884cc4ced9525dea6ff8dda"
server_key_version: "encryptionkey@0"
I20260808 18:26:51.848726 13843 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:26:51.851065 13858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:51.852270 13843 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:26:51.852404 13843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "b622bd0cc4054d349e3e70bd979b0142"
format_stamp: "Formatted at 2026-08-08 18:26:51 on dist-test-slave-5k54"
server_key: "e407e3001609629cc35547854ad8207a"
server_key_iv: "ab30f37a7884cc4ced9525dea6ff8dda"
server_key_version: "encryptionkey@0"
I20260808 18:26:51.852521 13843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:26:51.857152 13843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:26:51.860139 13843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:26:51.860330 13843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:26:51.867818 13843 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40657
I20260808 18:26:51.867918 13915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40657 every 8 connection(s)
I20260808 18:26:51.868848 13843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:26:51.869498 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 13843
I20260808 18:26:51.869686 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:26:51.870019 23842 external_mini_cluster.cc:1428] Setting key ce2dc92a3c2348b6e97f6daf60f20a50
I20260808 18:26:51.872325 13916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:26:51.875958 13919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:51.871573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50417 (local address 127.0.0.1:40657)
0808 18:26:51.872005 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:51.872027 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:51.872073 (+ 46us) server_negotiation.cc:407] Connection header received
0808 18:26:51.872965 (+ 892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:51.872988 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:51.873355 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:51.873741 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:51.874446 (+ 705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:51.874452 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:51.874468 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:26:51.874479 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:51.874539 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:26:51.874567 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:51.874716 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:51.874787 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 18:26:51.874942 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":124,"threads_started":1}
I20260808 18:26:51.878939 23842 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:26:51.879725 13916 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b622bd0cc4054d349e3e70bd979b0142: Bootstrap starting.
I20260808 18:26:51.882138 13916 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b622bd0cc4054d349e3e70bd979b0142: Neither blocks nor log segments found. Creating new log.
I20260808 18:26:51.882977 13919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:51.881182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45393 (local address 127.0.0.1:40657)
0808 18:26:51.881320 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:51.881327 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:51.881527 (+ 200us) server_negotiation.cc:407] Connection header received
0808 18:26:51.881675 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:51.881680 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:51.881718 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:51.881804 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:51.882574 (+ 770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:51.882577 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:51.882579 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:26:51.882590 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:51.882615 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:26:51.882626 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:51.882699 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:51.882813 (+ 114us) server_negotiation.cc:299] Negotiation successful
0808 18:26:51.882843 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 18:26:51.883167 13916 log.cc:824] T 00000000000000000000000000000000 P b622bd0cc4054d349e3e70bd979b0142: Log is configured to *not* fsync() on all Append() calls
I20260808 18:26:51.885258 13916 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b622bd0cc4054d349e3e70bd979b0142: No bootstrap required, opened a new log
I20260808 18:26:51.889113 13916 raft_consensus.cc:348] T 00000000000000000000000000000000 P b622bd0cc4054d349e3e70bd979b0142 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b622bd0cc4054d349e3e70bd979b0142" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40657 } }
I20260808 18:26:51.889377 13916 raft_consensus.cc:374] T 00000000000000000000000000000000 P b622bd0cc4054d349e3e70bd979b0142 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:26:51.889461 13916 raft_consensus.cc:729] T 00000000000000000000000000000000 P b622bd0cc4054d349e3e70bd979b0142 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b622bd0cc4054d349e3e70bd979b0142, State: Initialized, Role: FOLLOWER
I20260808 18:26:51.889957 13916 consensus_queue.cc:260] T 00000000000000000000000000000000 P b622bd0cc4054d349e3e70bd979b0142 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b622bd0cc4054d349e3e70bd979b0142" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40657 } }
I20260808 18:26:51.890151 13916 raft_consensus.cc:388] T 00000000000000000000000000000000 P b622bd0cc4054d349e3e70bd979b0142 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:26:51.890229 13916 raft_consensus.cc:482] T 00000000000000000000000000000000 P b622bd0cc4054d349e3e70bd979b0142 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:26:51.890311 13916 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b622bd0cc4054d349e3e70bd979b0142 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:26:51.891191 13916 raft_consensus.cc:504] T 00000000000000000000000000000000 P b622bd0cc4054d349e3e70bd979b0142 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b622bd0cc4054d349e3e70bd979b0142" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40657 } }
I20260808 18:26:51.891559 13916 leader_election.cc:302] T 00000000000000000000000000000000 P b622bd0cc4054d349e3e70bd979b0142 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b622bd0cc4054d349e3e70bd979b0142; no voters:
I20260808 18:26:51.891834 13916 leader_election.cc:288] T 00000000000000000000000000000000 P b622bd0cc4054d349e3e70bd979b0142 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:26:51.891959 13927 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b622bd0cc4054d349e3e70bd979b0142 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:26:51.892230 13927 raft_consensus.cc:686] T 00000000000000000000000000000000 P b622bd0cc4054d349e3e70bd979b0142 [term 1 LEADER]: Becoming Leader. State: Replica: b622bd0cc4054d349e3e70bd979b0142, State: Running, Role: LEADER
I20260808 18:26:51.892592 13927 consensus_queue.cc:237] T 00000000000000000000000000000000 P b622bd0cc4054d349e3e70bd979b0142 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b622bd0cc4054d349e3e70bd979b0142" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40657 } }
I20260808 18:26:51.893217 13916 sys_catalog.cc:564] T 00000000000000000000000000000000 P b622bd0cc4054d349e3e70bd979b0142 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:26:51.894331 13929 sys_catalog.cc:455] T 00000000000000000000000000000000 P b622bd0cc4054d349e3e70bd979b0142 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b622bd0cc4054d349e3e70bd979b0142. Latest consensus state: current_term: 1 leader_uuid: "b622bd0cc4054d349e3e70bd979b0142" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b622bd0cc4054d349e3e70bd979b0142" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40657 } } }
I20260808 18:26:51.894464 13929 sys_catalog.cc:458] T 00000000000000000000000000000000 P b622bd0cc4054d349e3e70bd979b0142 [sys.catalog]: This master's current role is: LEADER
I20260808 18:26:51.894884 13936 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:26:51.894845 13928 sys_catalog.cc:455] T 00000000000000000000000000000000 P b622bd0cc4054d349e3e70bd979b0142 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b622bd0cc4054d349e3e70bd979b0142" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b622bd0cc4054d349e3e70bd979b0142" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40657 } } }
I20260808 18:26:51.895709 13928 sys_catalog.cc:458] T 00000000000000000000000000000000 P b622bd0cc4054d349e3e70bd979b0142 [sys.catalog]: This master's current role is: LEADER
I20260808 18:26:51.898077 13936 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:26:51.903858 13936 catalog_manager.cc:1347] Generated new cluster ID: e0b7afa4fba74ce29d2df5dbda78d675
I20260808 18:26:51.903919 13936 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:26:51.913954 13936 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:26:51.914279 13936 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:26:51.930469 13936 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b622bd0cc4054d349e3e70bd979b0142: Generated new TSK 0
I20260808 18:26:51.931169 13936 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:26:51.940142 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 13843
2026-08-08T18:26:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1438 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213345431840-23842-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:26:51 dist-test-slave-5k54 krb5kdc[13951](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:26:52 dist-test-slave-5k54 krb5kdc[13951](info): set up 2 sockets
Aug 08 18:26:52 dist-test-slave-5k54 krb5kdc[13951](info): commencing operation
krb5kdc: starting...
W20260808 18:26:53.016813 23842 mini_kdc.cc:121] Time spent starting KDC: real 1.054s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Aug 08 18:26:53 dist-test-slave-5k54 krb5kdc[13951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:26:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:26:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213345431840-23842-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213345431840-23842-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 18:26:53.142635 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34635
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213345431840-23842-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34635
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:26:53.252148 13964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:26:53.252467 13964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:26:53.252552 13964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:26:53.256583 13964 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:26:53.256673 13964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:26:53.256712 13964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:26:53.256748 13964 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:26:53.256788 13964 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:26:53.262018 13964 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34635
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213345431840-23842-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:26:53.263131 13964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:26:53.264190 13964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:26:53.272689 13969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:53.272711 13972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:53.272737 13964 server_base.cc:1034] running on GCE node
W20260808 18:26:53.272684 13970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:53.273452 13964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:26:53.274578 13964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:26:53.275743 13964 hybrid_clock.cc:648] HybridClock initialized: now 1786213613275712 us; error 42 us; skew 500 ppm
Aug 08 18:26:53 dist-test-slave-5k54 krb5kdc[13951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213613, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:26:53.278570 13964 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 18:26:53.279520 13964 webserver.cc:460] Webserver started at http://127.0.0.1:40433/ using document root <none> and password file <none>
I20260808 18:26:53.280030 13964 fs_manager.cc:359] Metadata directory not provided
I20260808 18:26:53.280087 13964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:26:53.280249 13964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:26:53.282213 13964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "c4a18695f0374e86846bdee6e4dd254f"
format_stamp: "Formatted at 2026-08-08 18:26:53 on dist-test-slave-5k54"
server_key: "d8c7f8609489ca702009c1cf5f3d61f3"
server_key_iv: "bab071f0ad7ec93796ea960e7a3a28b1"
server_key_version: "encryptionkey@0"
I20260808 18:26:53.282729 13964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "c4a18695f0374e86846bdee6e4dd254f"
format_stamp: "Formatted at 2026-08-08 18:26:53 on dist-test-slave-5k54"
server_key: "d8c7f8609489ca702009c1cf5f3d61f3"
server_key_iv: "bab071f0ad7ec93796ea960e7a3a28b1"
server_key_version: "encryptionkey@0"
I20260808 18:26:53.286332 13964 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 18:26:53.288756 13979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:53.290020 13964 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:26:53.290158 13964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "c4a18695f0374e86846bdee6e4dd254f"
format_stamp: "Formatted at 2026-08-08 18:26:53 on dist-test-slave-5k54"
server_key: "d8c7f8609489ca702009c1cf5f3d61f3"
server_key_iv: "bab071f0ad7ec93796ea960e7a3a28b1"
server_key_version: "encryptionkey@0"
I20260808 18:26:53.290256 13964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:26:53.316089 13964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:26:53.319103 13964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:26:53.319314 13964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:26:53.326915 13964 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34635
I20260808 18:26:53.326952 14034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34635 every 8 connection(s)
I20260808 18:26:53.327913 13964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:26:53.329021 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 13964
I20260808 18:26:53.329185 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:26:53.329530 23842 external_mini_cluster.cc:1428] Setting key f2edd24abea3e05a0a23ebe575174bd9
I20260808 18:26:53.331210 14035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:26:53.336884 14035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4a18695f0374e86846bdee6e4dd254f: Bootstrap starting.
Aug 08 18:26:53 dist-test-slave-5k54 krb5kdc[13951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 18:26:53.339148 14035 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4a18695f0374e86846bdee6e4dd254f: Neither blocks nor log segments found. Creating new log.
I20260808 18:26:53.340098 14035 log.cc:824] T 00000000000000000000000000000000 P c4a18695f0374e86846bdee6e4dd254f: Log is configured to *not* fsync() on all Append() calls
I20260808 18:26:53.342121 14035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4a18695f0374e86846bdee6e4dd254f: No bootstrap required, opened a new log
I20260808 18:26:53.343894 14038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:53.331013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:40489 (local address 127.0.0.1:34635)
0808 18:26:53.331579 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:53.331596 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:53.331631 (+ 35us) server_negotiation.cc:407] Connection header received
0808 18:26:53.332262 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:53.332278 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:53.332631 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:53.332968 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:53.333787 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:53.334783 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:53.335727 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:53.336197 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:53.336361 (+ 164us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:53.338980 (+ 2619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:53.338984 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:53.338998 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:26:53.339025 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:53.340977 (+ 1952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:53.341481 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:53.341491 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:53.341495 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:53.341604 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:53.341938 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:53.341941 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:53.341943 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:53.342371 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:53.342593 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:53.342910 (+ 317us) server_negotiation.cc:299] Negotiation successful
0808 18:26:53.343135 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":154,"threads_started":1}
I20260808 18:26:53.346308 14035 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4a18695f0374e86846bdee6e4dd254f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4a18695f0374e86846bdee6e4dd254f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34635 } }
I20260808 18:26:53.346594 14035 raft_consensus.cc:374] T 00000000000000000000000000000000 P c4a18695f0374e86846bdee6e4dd254f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:26:53.346678 14035 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4a18695f0374e86846bdee6e4dd254f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4a18695f0374e86846bdee6e4dd254f, State: Initialized, Role: FOLLOWER
I20260808 18:26:53.347102 23842 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:26:53.347189 14035 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4a18695f0374e86846bdee6e4dd254f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4a18695f0374e86846bdee6e4dd254f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34635 } }
I20260808 18:26:53.347357 14035 raft_consensus.cc:388] T 00000000000000000000000000000000 P c4a18695f0374e86846bdee6e4dd254f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:26:53.347417 14035 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4a18695f0374e86846bdee6e4dd254f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:26:53.347537 14035 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4a18695f0374e86846bdee6e4dd254f [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:26:53.348811 14035 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4a18695f0374e86846bdee6e4dd254f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4a18695f0374e86846bdee6e4dd254f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34635 } }
I20260808 18:26:53.349171 14035 leader_election.cc:302] T 00000000000000000000000000000000 P c4a18695f0374e86846bdee6e4dd254f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4a18695f0374e86846bdee6e4dd254f; no voters:
I20260808 18:26:53.349550 14035 leader_election.cc:288] T 00000000000000000000000000000000 P c4a18695f0374e86846bdee6e4dd254f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:26:53.349728 14045 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4a18695f0374e86846bdee6e4dd254f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:26:53.349944 14045 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4a18695f0374e86846bdee6e4dd254f [term 1 LEADER]: Becoming Leader. State: Replica: c4a18695f0374e86846bdee6e4dd254f, State: Running, Role: LEADER
I20260808 18:26:53.350288 14045 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4a18695f0374e86846bdee6e4dd254f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4a18695f0374e86846bdee6e4dd254f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34635 } }
I20260808 18:26:53.350888 14035 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4a18695f0374e86846bdee6e4dd254f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:26:53.351610 14045 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4a18695f0374e86846bdee6e4dd254f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4a18695f0374e86846bdee6e4dd254f. Latest consensus state: current_term: 1 leader_uuid: "c4a18695f0374e86846bdee6e4dd254f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4a18695f0374e86846bdee6e4dd254f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34635 } } }
I20260808 18:26:53.352415 14045 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4a18695f0374e86846bdee6e4dd254f [sys.catalog]: This master's current role is: LEADER
I20260808 18:26:53.352288 14048 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4a18695f0374e86846bdee6e4dd254f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c4a18695f0374e86846bdee6e4dd254f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4a18695f0374e86846bdee6e4dd254f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34635 } } }
I20260808 18:26:53.353381 14048 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4a18695f0374e86846bdee6e4dd254f [sys.catalog]: This master's current role is: LEADER
W20260808 18:26:53.356638 14062 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c4a18695f0374e86846bdee6e4dd254f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:26:53.356721 14062 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:26:53.356802 14056 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:26:53.357798 14056 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:26:53.359365 14038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:53.349471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:59827 (local address 127.0.0.1:34635)
0808 18:26:53.349603 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:53.349608 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:53.349769 (+ 161us) server_negotiation.cc:407] Connection header received
0808 18:26:53.349938 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:53.349941 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:53.349992 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:53.350062 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:53.350823 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:53.351513 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:53.352660 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:53.352984 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:53.353095 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:53.355023 (+ 1928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:53.355027 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:53.355030 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:26:53.355053 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:53.356779 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:53.357382 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:53.357386 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:53.357388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:53.357476 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:53.357884 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:53.357887 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:53.357889 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:53.358111 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:53.358253 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:53.359026 (+ 773us) server_negotiation.cc:299] Negotiation successful
0808 18:26:53.359134 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 18:26:53.364717 14056 catalog_manager.cc:1347] Generated new cluster ID: 96c12917a3a949a59a524bd820022ecc
I20260808 18:26:53.364802 14056 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:26:53.383802 14056 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:26:53.384718 14056 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:26:53.391069 14056 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c4a18695f0374e86846bdee6e4dd254f: Generated new TSK 0
I20260808 18:26:53.391896 14056 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:26:53.396190 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 13964
W20260808 18:26:53.404109 13949 connection.cc:535] client connection to 127.0.0.1:34635 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34635): Connection reset by peer (error 104)
2026-08-08T18:26:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1455 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213345431840-23842-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:26:53 dist-test-slave-5k54 krb5kdc[14067](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:26:53 dist-test-slave-5k54 krb5kdc[14067](info): set up 2 sockets
Aug 08 18:26:53 dist-test-slave-5k54 krb5kdc[14067](info): commencing operation
krb5kdc: starting...
W20260808 18:26:54.476594 23842 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Aug 08 18:26:54 dist-test-slave-5k54 krb5kdc[14067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:26:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:26:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213345431840-23842-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213345431840-23842-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 18:26:54.610275 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39369
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213345431840-23842-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39369
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:26:54.725930 14081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:26:54.726207 14081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:26:54.726248 14081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:26:54.731015 14081 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:26:54.731119 14081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:26:54.731154 14081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:26:54.731189 14081 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:26:54.731217 14081 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:26:54.736982 14081 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39369
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213345431840-23842-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14081
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:26:54.738094 14081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:26:54.739135 14081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:26:54.747567 14091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:54.747560 14087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:54.747650 14081 server_base.cc:1034] running on GCE node
W20260808 18:26:54.747560 14088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:54.748551 14081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:26:54.749619 14081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:26:54.750810 14081 hybrid_clock.cc:648] HybridClock initialized: now 1786213614750796 us; error 31 us; skew 500 ppm
Aug 08 18:26:54 dist-test-slave-5k54 krb5kdc[14067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213614, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:26:54.763660 14081 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 18:26:54.764581 14081 webserver.cc:460] Webserver started at http://127.0.0.1:37257/ using document root <none> and password file <none>
I20260808 18:26:54.765131 14081 fs_manager.cc:359] Metadata directory not provided
I20260808 18:26:54.765192 14081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:26:54.765375 14081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:26:54.767549 14081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "481bb9345e4b4a5bb130cbcd70f7bb4b"
format_stamp: "Formatted at 2026-08-08 18:26:54 on dist-test-slave-5k54"
server_key: "6530c6501659e40146063ab3db33bc73"
server_key_iv: "fc5059c39814c9fc82b6c9ad4b1b6fde"
server_key_version: "encryptionkey@0"
I20260808 18:26:54.768227 14081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "481bb9345e4b4a5bb130cbcd70f7bb4b"
format_stamp: "Formatted at 2026-08-08 18:26:54 on dist-test-slave-5k54"
server_key: "6530c6501659e40146063ab3db33bc73"
server_key_iv: "fc5059c39814c9fc82b6c9ad4b1b6fde"
server_key_version: "encryptionkey@0"
I20260808 18:26:54.772133 14081 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 18:26:54.774540 14100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:54.775962 14081 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:26:54.776089 14081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "481bb9345e4b4a5bb130cbcd70f7bb4b"
format_stamp: "Formatted at 2026-08-08 18:26:54 on dist-test-slave-5k54"
server_key: "6530c6501659e40146063ab3db33bc73"
server_key_iv: "fc5059c39814c9fc82b6c9ad4b1b6fde"
server_key_version: "encryptionkey@0"
I20260808 18:26:54.776192 14081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:26:54.795516 14081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:26:54.798864 14081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:26:54.799067 14081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:26:54.806332 14081 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39369
I20260808 18:26:54.806347 14155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39369 every 8 connection(s)
I20260808 18:26:54.807350 14081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:26:54.809085 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 14081
I20260808 18:26:54.809262 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:26:54.809592 23842 external_mini_cluster.cc:1428] Setting key 4f1aec7a3c73ce2b6c2c1099f1199659
I20260808 18:26:54.810536 14156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:26:54.817411 14156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 481bb9345e4b4a5bb130cbcd70f7bb4b: Bootstrap starting.
I20260808 18:26:54.820047 14159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:54.811100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:45309 (local address 127.0.0.1:39369)
0808 18:26:54.811551 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:54.811570 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:54.811612 (+ 42us) server_negotiation.cc:407] Connection header received
0808 18:26:54.812464 (+ 852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:54.812485 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:54.812911 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:54.813278 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:54.813705 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:54.815188 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:54.816179 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:54.816872 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:54.817032 (+ 160us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:54.817977 (+ 945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:54.817983 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:54.817998 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:26:54.818498 (+ 500us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:54.818574 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:26:54.818608 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:54.818775 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:54.818957 (+ 182us) server_negotiation.cc:299] Negotiation successful
0808 18:26:54.819091 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":124,"threads_started":1}
I20260808 18:26:54.820217 14156 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 481bb9345e4b4a5bb130cbcd70f7bb4b: Neither blocks nor log segments found. Creating new log.
I20260808 18:26:54.821096 14156 log.cc:824] T 00000000000000000000000000000000 P 481bb9345e4b4a5bb130cbcd70f7bb4b: Log is configured to *not* fsync() on all Append() calls
I20260808 18:26:54.823246 23842 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:26:54.823462 14156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 481bb9345e4b4a5bb130cbcd70f7bb4b: No bootstrap required, opened a new log
I20260808 18:26:54.827755 14156 raft_consensus.cc:348] T 00000000000000000000000000000000 P 481bb9345e4b4a5bb130cbcd70f7bb4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "481bb9345e4b4a5bb130cbcd70f7bb4b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39369 } }
I20260808 18:26:54.828051 14156 raft_consensus.cc:374] T 00000000000000000000000000000000 P 481bb9345e4b4a5bb130cbcd70f7bb4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:26:54.828111 14156 raft_consensus.cc:729] T 00000000000000000000000000000000 P 481bb9345e4b4a5bb130cbcd70f7bb4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 481bb9345e4b4a5bb130cbcd70f7bb4b, State: Initialized, Role: FOLLOWER
I20260808 18:26:54.828562 14156 consensus_queue.cc:260] T 00000000000000000000000000000000 P 481bb9345e4b4a5bb130cbcd70f7bb4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "481bb9345e4b4a5bb130cbcd70f7bb4b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39369 } }
I20260808 18:26:54.828718 14156 raft_consensus.cc:388] T 00000000000000000000000000000000 P 481bb9345e4b4a5bb130cbcd70f7bb4b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:26:54.828807 14156 raft_consensus.cc:482] T 00000000000000000000000000000000 P 481bb9345e4b4a5bb130cbcd70f7bb4b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:26:54.828923 14156 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 481bb9345e4b4a5bb130cbcd70f7bb4b [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:26:54.829675 14159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:54.825397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:55593 (local address 127.0.0.1:39369)
0808 18:26:54.825530 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:54.825535 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:54.825654 (+ 119us) server_negotiation.cc:407] Connection header received
0808 18:26:54.825751 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:54.825755 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:54.825811 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:54.825892 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:54.826142 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:54.826921 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:54.827870 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:54.828214 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:54.828333 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:54.829227 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:54.829234 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:54.829240 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:26:54.829256 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:54.829294 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:26:54.829308 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:54.829395 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:54.829505 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 18:26:54.829539 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 18:26:54.830199 14156 raft_consensus.cc:504] T 00000000000000000000000000000000 P 481bb9345e4b4a5bb130cbcd70f7bb4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "481bb9345e4b4a5bb130cbcd70f7bb4b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39369 } }
I20260808 18:26:54.830557 14156 leader_election.cc:302] T 00000000000000000000000000000000 P 481bb9345e4b4a5bb130cbcd70f7bb4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 481bb9345e4b4a5bb130cbcd70f7bb4b; no voters:
I20260808 18:26:54.830907 14156 leader_election.cc:288] T 00000000000000000000000000000000 P 481bb9345e4b4a5bb130cbcd70f7bb4b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:26:54.831027 14167 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 481bb9345e4b4a5bb130cbcd70f7bb4b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:26:54.831288 14167 raft_consensus.cc:686] T 00000000000000000000000000000000 P 481bb9345e4b4a5bb130cbcd70f7bb4b [term 1 LEADER]: Becoming Leader. State: Replica: 481bb9345e4b4a5bb130cbcd70f7bb4b, State: Running, Role: LEADER
I20260808 18:26:54.831661 14167 consensus_queue.cc:237] T 00000000000000000000000000000000 P 481bb9345e4b4a5bb130cbcd70f7bb4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "481bb9345e4b4a5bb130cbcd70f7bb4b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39369 } }
I20260808 18:26:54.832154 14156 sys_catalog.cc:564] T 00000000000000000000000000000000 P 481bb9345e4b4a5bb130cbcd70f7bb4b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:26:54.833397 14168 sys_catalog.cc:455] T 00000000000000000000000000000000 P 481bb9345e4b4a5bb130cbcd70f7bb4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "481bb9345e4b4a5bb130cbcd70f7bb4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "481bb9345e4b4a5bb130cbcd70f7bb4b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39369 } } }
I20260808 18:26:54.833400 14169 sys_catalog.cc:455] T 00000000000000000000000000000000 P 481bb9345e4b4a5bb130cbcd70f7bb4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 481bb9345e4b4a5bb130cbcd70f7bb4b. Latest consensus state: current_term: 1 leader_uuid: "481bb9345e4b4a5bb130cbcd70f7bb4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "481bb9345e4b4a5bb130cbcd70f7bb4b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39369 } } }
I20260808 18:26:54.833590 14169 sys_catalog.cc:458] T 00000000000000000000000000000000 P 481bb9345e4b4a5bb130cbcd70f7bb4b [sys.catalog]: This master's current role is: LEADER
I20260808 18:26:54.833909 14168 sys_catalog.cc:458] T 00000000000000000000000000000000 P 481bb9345e4b4a5bb130cbcd70f7bb4b [sys.catalog]: This master's current role is: LEADER
I20260808 18:26:54.834600 14180 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:26:54.837397 14180 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:26:54.843299 14180 catalog_manager.cc:1347] Generated new cluster ID: e0121ffb1e294c4a9a84a9b3d63f5a11
I20260808 18:26:54.843377 14180 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:26:54.862123 14180 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:26:54.863412 14180 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:26:54.875322 14180 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 481bb9345e4b4a5bb130cbcd70f7bb4b: Generated new TSK 0
I20260808 18:26:54.876152 14180 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:26:54.887075 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 14081
W20260808 18:26:54.893261 14065 connection.cc:535] client connection to 127.0.0.1:39369 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39369): Connection reset by peer (error 104)
2026-08-08T18:26:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1488 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213345431840-23842-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:26:54 dist-test-slave-5k54 krb5kdc[14191](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:26:54 dist-test-slave-5k54 krb5kdc[14191](info): set up 2 sockets
Aug 08 18:26:54 dist-test-slave-5k54 krb5kdc[14191](info): commencing operation
krb5kdc: starting...
W20260808 18:26:55.975975 23842 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Aug 08 18:26:56 dist-test-slave-5k54 krb5kdc[14191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213616, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:26:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:26:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213345431840-23842-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213345431840-23842-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 18:26:56.100695 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38993
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213345431840-23842-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38993
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:26:56.211958 14206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:26:56.212214 14206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:26:56.212311 14206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:26:56.216401 14206 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:26:56.216487 14206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:26:56.216516 14206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:26:56.216553 14206 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:26:56.216585 14206 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:26:56.221743 14206 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38993
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213345431840-23842-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14206
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:26:56.222738 14206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:26:56.223795 14206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:26:56.232506 14211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:56.232497 14214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:56.232497 14212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:56.232928 14206 server_base.cc:1034] running on GCE node
I20260808 18:26:56.233500 14206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:26:56.234573 14206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:26:56.235805 14206 hybrid_clock.cc:648] HybridClock initialized: now 1786213616235748 us; error 33 us; skew 500 ppm
Aug 08 18:26:56 dist-test-slave-5k54 krb5kdc[14191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213616, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:26:56.238657 14206 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 18:26:56.239555 14206 webserver.cc:460] Webserver started at http://127.0.0.1:41837/ using document root <none> and password file <none>
I20260808 18:26:56.240082 14206 fs_manager.cc:359] Metadata directory not provided
I20260808 18:26:56.240144 14206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:26:56.240317 14206 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:26:56.242321 14206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "1ca62d7c0c504551ab121d59438472b2"
format_stamp: "Formatted at 2026-08-08 18:26:56 on dist-test-slave-5k54"
server_key: "e1d1a8743fe706440372fec950b704e0"
server_key_iv: "826c845be885acbf4e08ecb70f2b2652"
server_key_version: "encryptionkey@0"
I20260808 18:26:56.242846 14206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "1ca62d7c0c504551ab121d59438472b2"
format_stamp: "Formatted at 2026-08-08 18:26:56 on dist-test-slave-5k54"
server_key: "e1d1a8743fe706440372fec950b704e0"
server_key_iv: "826c845be885acbf4e08ecb70f2b2652"
server_key_version: "encryptionkey@0"
I20260808 18:26:56.246501 14206 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260808 18:26:56.249051 14221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:56.250314 14206 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:26:56.250474 14206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "1ca62d7c0c504551ab121d59438472b2"
format_stamp: "Formatted at 2026-08-08 18:26:56 on dist-test-slave-5k54"
server_key: "e1d1a8743fe706440372fec950b704e0"
server_key_iv: "826c845be885acbf4e08ecb70f2b2652"
server_key_version: "encryptionkey@0"
I20260808 18:26:56.250584 14206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:26:56.255069 14206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:26:56.258133 14206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:26:56.258317 14206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:26:56.265362 14206 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38993
I20260808 18:26:56.265396 14278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38993 every 8 connection(s)
I20260808 18:26:56.266301 14206 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:26:56.267403 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 14206
I20260808 18:26:56.267601 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:26:56.267904 23842 external_mini_cluster.cc:1428] Setting key cbfb825e15cd2c6e2958d4e37a9d2eca
I20260808 18:26:56.269352 14279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:26:56.273543 14282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:56.269293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:33435 (local address 127.0.0.1:38993)
0808 18:26:56.269715 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:56.269735 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:56.269778 (+ 43us) server_negotiation.cc:407] Connection header received
0808 18:26:56.270592 (+ 814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:56.270614 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:56.270961 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:56.271313 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:56.272128 (+ 815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:56.272134 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:56.272149 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:26:56.272161 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:56.272225 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:26:56.272251 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:56.272401 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:56.272470 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 18:26:56.272623 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":135,"threads_started":1}
I20260808 18:26:56.276254 23842 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:26:56.276355 14279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ca62d7c0c504551ab121d59438472b2: Bootstrap starting.
I20260808 18:26:56.278733 14279 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ca62d7c0c504551ab121d59438472b2: Neither blocks nor log segments found. Creating new log.
I20260808 18:26:56.279707 14279 log.cc:824] T 00000000000000000000000000000000 P 1ca62d7c0c504551ab121d59438472b2: Log is configured to *not* fsync() on all Append() calls
I20260808 18:26:56.279910 14282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:56.278235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:48449 (local address 127.0.0.1:38993)
0808 18:26:56.278336 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:56.278339 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:56.278579 (+ 240us) server_negotiation.cc:407] Connection header received
0808 18:26:56.278721 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:56.278723 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:56.278750 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:56.278830 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:56.279518 (+ 688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:56.279521 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:56.279523 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:26:56.279533 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:56.279556 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:26:56.279565 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:56.279628 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:56.279726 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 18:26:56.279755 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 18:26:56.281654 14279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ca62d7c0c504551ab121d59438472b2: No bootstrap required, opened a new log
I20260808 18:26:56.285106 14279 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ca62d7c0c504551ab121d59438472b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ca62d7c0c504551ab121d59438472b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38993 } }
I20260808 18:26:56.285369 14279 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ca62d7c0c504551ab121d59438472b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:26:56.285439 14279 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ca62d7c0c504551ab121d59438472b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca62d7c0c504551ab121d59438472b2, State: Initialized, Role: FOLLOWER
I20260808 18:26:56.285903 14279 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ca62d7c0c504551ab121d59438472b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ca62d7c0c504551ab121d59438472b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38993 } }
I20260808 18:26:56.286041 14279 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ca62d7c0c504551ab121d59438472b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:26:56.286105 14279 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ca62d7c0c504551ab121d59438472b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:26:56.286193 14279 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ca62d7c0c504551ab121d59438472b2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:26:56.287050 14279 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ca62d7c0c504551ab121d59438472b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ca62d7c0c504551ab121d59438472b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38993 } }
I20260808 18:26:56.287371 14279 leader_election.cc:302] T 00000000000000000000000000000000 P 1ca62d7c0c504551ab121d59438472b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ca62d7c0c504551ab121d59438472b2; no voters:
I20260808 18:26:56.287705 14279 leader_election.cc:288] T 00000000000000000000000000000000 P 1ca62d7c0c504551ab121d59438472b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:26:56.287854 14292 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ca62d7c0c504551ab121d59438472b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:26:56.288076 14292 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ca62d7c0c504551ab121d59438472b2 [term 1 LEADER]: Becoming Leader. State: Replica: 1ca62d7c0c504551ab121d59438472b2, State: Running, Role: LEADER
I20260808 18:26:56.288430 14292 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ca62d7c0c504551ab121d59438472b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ca62d7c0c504551ab121d59438472b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38993 } }
I20260808 18:26:56.288892 14279 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ca62d7c0c504551ab121d59438472b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:26:56.290009 14293 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ca62d7c0c504551ab121d59438472b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ca62d7c0c504551ab121d59438472b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ca62d7c0c504551ab121d59438472b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38993 } } }
I20260808 18:26:56.290135 14293 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ca62d7c0c504551ab121d59438472b2 [sys.catalog]: This master's current role is: LEADER
I20260808 18:26:56.290463 14294 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ca62d7c0c504551ab121d59438472b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ca62d7c0c504551ab121d59438472b2. Latest consensus state: current_term: 1 leader_uuid: "1ca62d7c0c504551ab121d59438472b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ca62d7c0c504551ab121d59438472b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38993 } } }
I20260808 18:26:56.290537 14301 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:26:56.290623 14294 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ca62d7c0c504551ab121d59438472b2 [sys.catalog]: This master's current role is: LEADER
I20260808 18:26:56.293641 14301 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:26:56.298950 14301 catalog_manager.cc:1347] Generated new cluster ID: b46e0c7204b54455892990d32aed3525
I20260808 18:26:56.299014 14301 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:26:56.312817 14301 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:26:56.313145 14301 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:26:56.328572 14301 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ca62d7c0c504551ab121d59438472b2: Generated new TSK 0
I20260808 18:26:56.329391 14301 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:26:56.341091 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 14206
2026-08-08T18:26:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1455 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213345431840-23842-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:26:56 dist-test-slave-5k54 krb5kdc[14314](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:26:56 dist-test-slave-5k54 krb5kdc[14314](info): set up 2 sockets
Aug 08 18:26:56 dist-test-slave-5k54 krb5kdc[14314](info): commencing operation
krb5kdc: starting...
W20260808 18:26:57.423940 23842 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Aug 08 18:26:57 dist-test-slave-5k54 krb5kdc[14314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:26:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:26:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213345431840-23842-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213345431840-23842-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 18:26:57.549849 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46509
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213345431840-23842-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46509
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:26:57.659818 14327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:26:57.660068 14327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:26:57.660132 14327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:26:57.664042 14327 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:26:57.664101 14327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:26:57.664120 14327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:26:57.664162 14327 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:26:57.664194 14327 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:26:57.669370 14327 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46509
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213345431840-23842-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:26:57.670358 14327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:26:57.671332 14327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:26:57.679710 14333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:57.679759 14327 server_base.cc:1034] running on GCE node
W20260808 18:26:57.679723 14335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:57.679705 14332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:57.680410 14327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:26:57.681427 14327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:26:57.682588 14327 hybrid_clock.cc:648] HybridClock initialized: now 1786213617682563 us; error 36 us; skew 500 ppm
Aug 08 18:26:57 dist-test-slave-5k54 krb5kdc[14314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213617, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:26:57.685509 14327 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 18:26:57.686383 14327 webserver.cc:460] Webserver started at http://127.0.0.1:42701/ using document root <none> and password file <none>
I20260808 18:26:57.686925 14327 fs_manager.cc:359] Metadata directory not provided
I20260808 18:26:57.687014 14327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:26:57.687243 14327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:26:57.689272 14327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "d0b09f3688da4213bb275ad6adbbd8c9"
format_stamp: "Formatted at 2026-08-08 18:26:57 on dist-test-slave-5k54"
server_key: "ca984922b348d2ba7ab203b5752e590f"
server_key_iv: "f6957292a66f05d0355f403a18de0868"
server_key_version: "encryptionkey@0"
I20260808 18:26:57.689845 14327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "d0b09f3688da4213bb275ad6adbbd8c9"
format_stamp: "Formatted at 2026-08-08 18:26:57 on dist-test-slave-5k54"
server_key: "ca984922b348d2ba7ab203b5752e590f"
server_key_iv: "f6957292a66f05d0355f403a18de0868"
server_key_version: "encryptionkey@0"
I20260808 18:26:57.693675 14327 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260808 18:26:57.696183 14344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:57.697397 14327 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:26:57.697492 14327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "d0b09f3688da4213bb275ad6adbbd8c9"
format_stamp: "Formatted at 2026-08-08 18:26:57 on dist-test-slave-5k54"
server_key: "ca984922b348d2ba7ab203b5752e590f"
server_key_iv: "f6957292a66f05d0355f403a18de0868"
server_key_version: "encryptionkey@0"
I20260808 18:26:57.697571 14327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:26:57.711557 14327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:26:57.714593 14327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:26:57.714740 14327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:26:57.722069 14327 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46509
I20260808 18:26:57.722095 14401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46509 every 8 connection(s)
I20260808 18:26:57.722954 14327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:26:57.725651 14402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:26:57.726159 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 14327
I20260808 18:26:57.726318 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:26:57.726647 23842 external_mini_cluster.cc:1428] Setting key e0b263089962f8905098299f5f047325
I20260808 18:26:57.732440 14402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0b09f3688da4213bb275ad6adbbd8c9: Bootstrap starting.
I20260808 18:26:57.734925 14402 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0b09f3688da4213bb275ad6adbbd8c9: Neither blocks nor log segments found. Creating new log.
I20260808 18:26:57.735767 14402 log.cc:824] T 00000000000000000000000000000000 P d0b09f3688da4213bb275ad6adbbd8c9: Log is configured to *not* fsync() on all Append() calls
Aug 08 18:26:57 dist-test-slave-5k54 krb5kdc[14314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 18:26:57.737785 14402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0b09f3688da4213bb275ad6adbbd8c9: No bootstrap required, opened a new log
I20260808 18:26:57.741978 14405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:57.728040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:36069 (local address 127.0.0.1:46509)
0808 18:26:57.728448 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:57.728467 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:57.728512 (+ 45us) server_negotiation.cc:407] Connection header received
0808 18:26:57.729353 (+ 841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:57.729376 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:57.729807 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:57.730168 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:57.730827 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:57.732327 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:57.733216 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:57.733946 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:57.734106 (+ 160us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:57.737019 (+ 2913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:57.737024 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:57.737037 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:26:57.737063 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:57.738841 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:57.739407 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:57.739411 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:57.739415 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:57.739575 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:57.739986 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:57.739989 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:57.739990 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:57.740361 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:57.740511 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:57.740843 (+ 332us) server_negotiation.cc:299] Negotiation successful
0808 18:26:57.741084 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":113,"threads_started":1}
I20260808 18:26:57.742111 14402 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0b09f3688da4213bb275ad6adbbd8c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0b09f3688da4213bb275ad6adbbd8c9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46509 } }
I20260808 18:26:57.742424 14402 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0b09f3688da4213bb275ad6adbbd8c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:26:57.742498 14402 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0b09f3688da4213bb275ad6adbbd8c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0b09f3688da4213bb275ad6adbbd8c9, State: Initialized, Role: FOLLOWER
I20260808 18:26:57.743005 14402 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0b09f3688da4213bb275ad6adbbd8c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0b09f3688da4213bb275ad6adbbd8c9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46509 } }
I20260808 18:26:57.743203 14402 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0b09f3688da4213bb275ad6adbbd8c9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:26:57.743299 14402 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0b09f3688da4213bb275ad6adbbd8c9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:26:57.743487 14402 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0b09f3688da4213bb275ad6adbbd8c9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:26:57.744622 14402 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0b09f3688da4213bb275ad6adbbd8c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0b09f3688da4213bb275ad6adbbd8c9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46509 } }
I20260808 18:26:57.744978 14402 leader_election.cc:302] T 00000000000000000000000000000000 P d0b09f3688da4213bb275ad6adbbd8c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0b09f3688da4213bb275ad6adbbd8c9; no voters:
I20260808 18:26:57.745368 14402 leader_election.cc:288] T 00000000000000000000000000000000 P d0b09f3688da4213bb275ad6adbbd8c9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:26:57.745461 14407 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0b09f3688da4213bb275ad6adbbd8c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:26:57.745663 14407 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0b09f3688da4213bb275ad6adbbd8c9 [term 1 LEADER]: Becoming Leader. State: Replica: d0b09f3688da4213bb275ad6adbbd8c9, State: Running, Role: LEADER
I20260808 18:26:57.745810 23842 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:26:57.745978 14407 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0b09f3688da4213bb275ad6adbbd8c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0b09f3688da4213bb275ad6adbbd8c9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46509 } }
I20260808 18:26:57.746562 14402 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0b09f3688da4213bb275ad6adbbd8c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:26:57.747143 14407 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0b09f3688da4213bb275ad6adbbd8c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0b09f3688da4213bb275ad6adbbd8c9. Latest consensus state: current_term: 1 leader_uuid: "d0b09f3688da4213bb275ad6adbbd8c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0b09f3688da4213bb275ad6adbbd8c9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46509 } } }
I20260808 18:26:57.747273 14407 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0b09f3688da4213bb275ad6adbbd8c9 [sys.catalog]: This master's current role is: LEADER
I20260808 18:26:57.747673 14418 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:26:57.747709 14408 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0b09f3688da4213bb275ad6adbbd8c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0b09f3688da4213bb275ad6adbbd8c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0b09f3688da4213bb275ad6adbbd8c9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46509 } } }
I20260808 18:26:57.747948 14408 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0b09f3688da4213bb275ad6adbbd8c9 [sys.catalog]: This master's current role is: LEADER
I20260808 18:26:57.750943 14418 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:26:57.757009 14418 catalog_manager.cc:1347] Generated new cluster ID: a70e7c7c8b564c78a878d1dad8aae398
I20260808 18:26:57.757071 14418 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:26:57.758256 14405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:57.748099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:60271 (local address 127.0.0.1:46509)
0808 18:26:57.748201 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:57.748205 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:57.749039 (+ 834us) server_negotiation.cc:407] Connection header received
0808 18:26:57.749225 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:57.749228 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:57.749276 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:57.749336 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:57.750271 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:57.750792 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:57.752002 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:57.752331 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:57.752452 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:57.754228 (+ 1776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:57.754231 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:57.754234 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:26:57.754259 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:57.756221 (+ 1962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:57.756768 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:57.756772 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:57.756773 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:57.756865 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:26:57.757284 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:26:57.757288 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:26:57.757290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:26:57.757512 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:57.757663 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:57.757983 (+ 320us) server_negotiation.cc:299] Negotiation successful
0808 18:26:57.758100 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 18:26:57.767693 14418 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:26:57.768842 14418 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:26:57.785367 14418 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0b09f3688da4213bb275ad6adbbd8c9: Generated new TSK 0
I20260808 18:26:57.786036 14418 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:26:57.795010 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 14327
W20260808 18:26:57.803550 14312 connection.cc:535] client connection to 127.0.0.1:46509 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46509): Connection reset by peer (error 104)
2026-08-08T18:26:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1455 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213345431840-23842-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:26:57 dist-test-slave-5k54 krb5kdc[14434](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:26:57 dist-test-slave-5k54 krb5kdc[14434](info): set up 2 sockets
Aug 08 18:26:57 dist-test-slave-5k54 krb5kdc[14434](info): commencing operation
krb5kdc: starting...
W20260808 18:26:58.868158 23842 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Aug 08 18:26:58 dist-test-slave-5k54 krb5kdc[14434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:26:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:26:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213345431840-23842-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213345431840-23842-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 18:26:58.994817 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36245
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213345431840-23842-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36245
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:26:59.103948 14447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:26:59.104256 14447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:26:59.104342 14447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:26:59.108413 14447 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:26:59.108502 14447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:26:59.108538 14447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:26:59.108592 14447 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:26:59.108628 14447 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:26:59.113940 14447 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36245
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213345431840-23842-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14447
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:26:59.115061 14447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:26:59.116168 14447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:26:59.124541 14455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:59.124569 14453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:26:59.124560 14452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:26:59.124796 14447 server_base.cc:1034] running on GCE node
I20260808 18:26:59.125504 14447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:26:59.126685 14447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:26:59.127892 14447 hybrid_clock.cc:648] HybridClock initialized: now 1786213619127858 us; error 44 us; skew 500 ppm
Aug 08 18:26:59 dist-test-slave-5k54 krb5kdc[14434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213619, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:26:59.130728 14447 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 18:26:59.131623 14447 webserver.cc:460] Webserver started at http://127.0.0.1:37319/ using document root <none> and password file <none>
I20260808 18:26:59.132136 14447 fs_manager.cc:359] Metadata directory not provided
I20260808 18:26:59.132194 14447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:26:59.132371 14447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:26:59.134369 14447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "39d557d53d17431facd1ad0e794b51a3"
format_stamp: "Formatted at 2026-08-08 18:26:59 on dist-test-slave-5k54"
server_key: "8d91989f430e14baff90f7dad1fd9955"
server_key_iv: "426eed13497c24e5daefd4f3f3c21bc1"
server_key_version: "encryptionkey@0"
I20260808 18:26:59.134917 14447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "39d557d53d17431facd1ad0e794b51a3"
format_stamp: "Formatted at 2026-08-08 18:26:59 on dist-test-slave-5k54"
server_key: "8d91989f430e14baff90f7dad1fd9955"
server_key_iv: "426eed13497c24e5daefd4f3f3c21bc1"
server_key_version: "encryptionkey@0"
I20260808 18:26:59.138727 14447 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260808 18:26:59.141395 14462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:26:59.142717 14447 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:26:59.142906 14447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "39d557d53d17431facd1ad0e794b51a3"
format_stamp: "Formatted at 2026-08-08 18:26:59 on dist-test-slave-5k54"
server_key: "8d91989f430e14baff90f7dad1fd9955"
server_key_iv: "426eed13497c24e5daefd4f3f3c21bc1"
server_key_version: "encryptionkey@0"
I20260808 18:26:59.143029 14447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:26:59.154816 14447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:26:59.157812 14447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:26:59.158033 14447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:26:59.165711 14447 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36245
I20260808 18:26:59.165720 14519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36245 every 8 connection(s)
I20260808 18:26:59.166712 14447 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:26:59.169363 14520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:26:59.171384 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 14447
I20260808 18:26:59.171579 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:26:59.171916 23842 external_mini_cluster.cc:1428] Setting key a7bbb2b569243e90d5baddf0fbd7b37f
I20260808 18:26:59.175474 14520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39d557d53d17431facd1ad0e794b51a3: Bootstrap starting.
I20260808 18:26:59.177956 14520 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 39d557d53d17431facd1ad0e794b51a3: Neither blocks nor log segments found. Creating new log.
I20260808 18:26:59.178956 14520 log.cc:824] T 00000000000000000000000000000000 P 39d557d53d17431facd1ad0e794b51a3: Log is configured to *not* fsync() on all Append() calls
I20260808 18:26:59.181183 14520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39d557d53d17431facd1ad0e794b51a3: No bootstrap required, opened a new log
I20260808 18:26:59.181782 14523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:59.173307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:43459 (local address 127.0.0.1:36245)
0808 18:26:59.173749 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:59.173766 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:59.173810 (+ 44us) server_negotiation.cc:407] Connection header received
0808 18:26:59.174518 (+ 708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:59.174534 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:59.174897 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:59.175182 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:59.175598 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:59.176672 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:59.177878 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:59.178702 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:59.178927 (+ 225us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:59.180034 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:59.180039 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:59.180051 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:26:59.180422 (+ 371us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:59.180476 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:26:59.180497 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:59.180603 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:59.180864 (+ 261us) server_negotiation.cc:299] Negotiation successful
0808 18:26:59.181008 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":139,"threads_started":1}
I20260808 18:26:59.184855 23842 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:26:59.185266 14520 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39d557d53d17431facd1ad0e794b51a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39d557d53d17431facd1ad0e794b51a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36245 } }
I20260808 18:26:59.185542 14520 raft_consensus.cc:374] T 00000000000000000000000000000000 P 39d557d53d17431facd1ad0e794b51a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:26:59.185601 14520 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39d557d53d17431facd1ad0e794b51a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39d557d53d17431facd1ad0e794b51a3, State: Initialized, Role: FOLLOWER
I20260808 18:26:59.186075 14520 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39d557d53d17431facd1ad0e794b51a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39d557d53d17431facd1ad0e794b51a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36245 } }
I20260808 18:26:59.186305 14520 raft_consensus.cc:388] T 00000000000000000000000000000000 P 39d557d53d17431facd1ad0e794b51a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:26:59.186417 14520 raft_consensus.cc:482] T 00000000000000000000000000000000 P 39d557d53d17431facd1ad0e794b51a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:26:59.186528 14520 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39d557d53d17431facd1ad0e794b51a3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:26:59.187664 14520 raft_consensus.cc:504] T 00000000000000000000000000000000 P 39d557d53d17431facd1ad0e794b51a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39d557d53d17431facd1ad0e794b51a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36245 } }
I20260808 18:26:59.188072 14520 leader_election.cc:302] T 00000000000000000000000000000000 P 39d557d53d17431facd1ad0e794b51a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 39d557d53d17431facd1ad0e794b51a3; no voters:
I20260808 18:26:59.188432 14520 leader_election.cc:288] T 00000000000000000000000000000000 P 39d557d53d17431facd1ad0e794b51a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:26:59.188509 14530 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 39d557d53d17431facd1ad0e794b51a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:26:59.188721 14530 raft_consensus.cc:686] T 00000000000000000000000000000000 P 39d557d53d17431facd1ad0e794b51a3 [term 1 LEADER]: Becoming Leader. State: Replica: 39d557d53d17431facd1ad0e794b51a3, State: Running, Role: LEADER
I20260808 18:26:59.189064 14530 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39d557d53d17431facd1ad0e794b51a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39d557d53d17431facd1ad0e794b51a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36245 } }
I20260808 18:26:59.189680 14520 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39d557d53d17431facd1ad0e794b51a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:26:59.190200 14530 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39d557d53d17431facd1ad0e794b51a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39d557d53d17431facd1ad0e794b51a3. Latest consensus state: current_term: 1 leader_uuid: "39d557d53d17431facd1ad0e794b51a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39d557d53d17431facd1ad0e794b51a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36245 } } }
I20260808 18:26:59.190981 14530 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39d557d53d17431facd1ad0e794b51a3 [sys.catalog]: This master's current role is: LEADER
I20260808 18:26:59.190862 14531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39d557d53d17431facd1ad0e794b51a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "39d557d53d17431facd1ad0e794b51a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39d557d53d17431facd1ad0e794b51a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36245 } } }
I20260808 18:26:59.191284 14531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39d557d53d17431facd1ad0e794b51a3 [sys.catalog]: This master's current role is: LEADER
I20260808 18:26:59.192425 14523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:26:59.187103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:34933 (local address 127.0.0.1:36245)
0808 18:26:59.187225 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 18:26:59.187229 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:26:59.187497 (+ 268us) server_negotiation.cc:407] Connection header received
0808 18:26:59.187649 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:26:59.187651 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:26:59.187690 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:26:59.187858 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:26:59.188083 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:59.188922 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:59.189996 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:26:59.190401 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:26:59.190513 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:26:59.191702 (+ 1189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:26:59.191706 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:26:59.191709 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:26:59.191723 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:26:59.191752 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:26:59.191768 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:26:59.191882 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 18:26:59.192247 (+ 365us) server_negotiation.cc:299] Negotiation successful
0808 18:26:59.192283 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 18:26:59.192543 14539 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:26:59.195480 14539 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:26:59.201236 14539 catalog_manager.cc:1347] Generated new cluster ID: 77cec788bc5849c9944d541120e8f01a
I20260808 18:26:59.201328 14539 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:26:59.216405 14539 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:26:59.217383 14539 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:26:59.228631 14539 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 39d557d53d17431facd1ad0e794b51a3: Generated new TSK 0
I20260808 18:26:59.229485 14539 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:26:59.241776 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 14447
W20260808 18:26:59.250567 14432 connection.cc:535] client connection to 127.0.0.1:36245 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36245): Connection reset by peer (error 104)
2026-08-08T18:26:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1447 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213345431840-23842-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:26:59 dist-test-slave-5k54 krb5kdc[14552](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:26:59 dist-test-slave-5k54 krb5kdc[14552](info): set up 2 sockets
Aug 08 18:26:59 dist-test-slave-5k54 krb5kdc[14552](info): commencing operation
krb5kdc: starting...
W20260808 18:27:00.312162 23842 mini_kdc.cc:121] Time spent starting KDC: real 1.046s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Aug 08 18:27:00 dist-test-slave-5k54 krb5kdc[14552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:27:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:27:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213345431840-23842-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213345431840-23842-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 18:27:00.439139 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37945
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213345431840-23842-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37945
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:00.554171 14567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:00.554467 14567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:00.554543 14567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:00.558554 14567 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:27:00.558638 14567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:00.558676 14567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:00.558732 14567 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:27:00.558770 14567 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:27:00.564141 14567 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37945
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213345431840-23842-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:00.565222 14567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:00.566591 14567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:00.575078 14573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:00.575070 14575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:00.575070 14572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:00.575475 14567 server_base.cc:1034] running on GCE node
I20260808 18:27:00.576046 14567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:00.577129 14567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:00.578361 14567 hybrid_clock.cc:648] HybridClock initialized: now 1786213620578326 us; error 45 us; skew 500 ppm
Aug 08 18:27:00 dist-test-slave-5k54 krb5kdc[14552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213620, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:00.586568 14567 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 18:27:00.587639 14567 webserver.cc:460] Webserver started at http://127.0.0.1:42051/ using document root <none> and password file <none>
I20260808 18:27:00.588214 14567 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:00.588305 14567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:00.588538 14567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:00.590602 14567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "305a0c62ced7456dadfbc6746f0b809b"
format_stamp: "Formatted at 2026-08-08 18:27:00 on dist-test-slave-5k54"
server_key: "88c0a80fda51d326a15db8ebbe7978f3"
server_key_iv: "963aca3c61e5edb9264ac4d3bc8c6141"
server_key_version: "encryptionkey@0"
I20260808 18:27:00.591177 14567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "305a0c62ced7456dadfbc6746f0b809b"
format_stamp: "Formatted at 2026-08-08 18:27:00 on dist-test-slave-5k54"
server_key: "88c0a80fda51d326a15db8ebbe7978f3"
server_key_iv: "963aca3c61e5edb9264ac4d3bc8c6141"
server_key_version: "encryptionkey@0"
I20260808 18:27:00.594879 14567 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 18:27:00.597347 14582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:00.598601 14567 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:27:00.598767 14567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "305a0c62ced7456dadfbc6746f0b809b"
format_stamp: "Formatted at 2026-08-08 18:27:00 on dist-test-slave-5k54"
server_key: "88c0a80fda51d326a15db8ebbe7978f3"
server_key_iv: "963aca3c61e5edb9264ac4d3bc8c6141"
server_key_version: "encryptionkey@0"
I20260808 18:27:00.598883 14567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:00.603518 14567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:00.606336 14567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:00.606546 14567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:00.613672 14567 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37945
I20260808 18:27:00.613695 14637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37945 every 8 connection(s)
I20260808 18:27:00.614542 14567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:27:00.617182 14638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:00.618485 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 14567
I20260808 18:27:00.618636 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:27:00.618978 23842 external_mini_cluster.cc:1428] Setting key a2ea8225f07bf90c8b7792c1945352d9
I20260808 18:27:00.623255 14638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 305a0c62ced7456dadfbc6746f0b809b: Bootstrap starting.
I20260808 18:27:00.624051 14642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:00.620396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:41411 (local address 127.0.0.1:37945)
0808 18:27:00.620823 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:00.620845 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:00.620879 (+ 34us) server_negotiation.cc:407] Connection header received
0808 18:27:00.621473 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:00.621487 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:00.621741 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:00.622069 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:00.622814 (+ 745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:00.622818 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:00.622829 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:27:00.622845 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:00.622897 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:27:00.622917 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:00.623082 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:00.623151 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 18:27:00.623303 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":147,"threads_started":1}
I20260808 18:27:00.625484 14638 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 305a0c62ced7456dadfbc6746f0b809b: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:00.626317 14638 log.cc:824] T 00000000000000000000000000000000 P 305a0c62ced7456dadfbc6746f0b809b: Log is configured to *not* fsync() on all Append() calls
I20260808 18:27:00.626674 23842 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:27:00.628397 14638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 305a0c62ced7456dadfbc6746f0b809b: No bootstrap required, opened a new log
I20260808 18:27:00.630298 14642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:00.628770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:59225 (local address 127.0.0.1:37945)
0808 18:27:00.628893 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:00.628898 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:00.629093 (+ 195us) server_negotiation.cc:407] Connection header received
0808 18:27:00.629196 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:00.629198 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:00.629225 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:00.629287 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:00.629947 (+ 660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:00.629949 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:00.629951 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:27:00.629960 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:00.629982 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:27:00.629991 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:00.630057 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:00.630147 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 18:27:00.630176 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 18:27:00.632475 14638 raft_consensus.cc:348] T 00000000000000000000000000000000 P 305a0c62ced7456dadfbc6746f0b809b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "305a0c62ced7456dadfbc6746f0b809b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37945 } }
I20260808 18:27:00.632705 14638 raft_consensus.cc:374] T 00000000000000000000000000000000 P 305a0c62ced7456dadfbc6746f0b809b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:00.632764 14638 raft_consensus.cc:729] T 00000000000000000000000000000000 P 305a0c62ced7456dadfbc6746f0b809b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 305a0c62ced7456dadfbc6746f0b809b, State: Initialized, Role: FOLLOWER
I20260808 18:27:00.633152 14638 consensus_queue.cc:260] T 00000000000000000000000000000000 P 305a0c62ced7456dadfbc6746f0b809b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "305a0c62ced7456dadfbc6746f0b809b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37945 } }
I20260808 18:27:00.633291 14638 raft_consensus.cc:388] T 00000000000000000000000000000000 P 305a0c62ced7456dadfbc6746f0b809b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:27:00.633370 14638 raft_consensus.cc:482] T 00000000000000000000000000000000 P 305a0c62ced7456dadfbc6746f0b809b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:27:00.633472 14638 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 305a0c62ced7456dadfbc6746f0b809b [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:00.634434 14638 raft_consensus.cc:504] T 00000000000000000000000000000000 P 305a0c62ced7456dadfbc6746f0b809b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "305a0c62ced7456dadfbc6746f0b809b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37945 } }
I20260808 18:27:00.634773 14638 leader_election.cc:302] T 00000000000000000000000000000000 P 305a0c62ced7456dadfbc6746f0b809b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 305a0c62ced7456dadfbc6746f0b809b; no voters:
I20260808 18:27:00.635059 14638 leader_election.cc:288] T 00000000000000000000000000000000 P 305a0c62ced7456dadfbc6746f0b809b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:27:00.635205 14650 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 305a0c62ced7456dadfbc6746f0b809b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:27:00.635469 14650 raft_consensus.cc:686] T 00000000000000000000000000000000 P 305a0c62ced7456dadfbc6746f0b809b [term 1 LEADER]: Becoming Leader. State: Replica: 305a0c62ced7456dadfbc6746f0b809b, State: Running, Role: LEADER
I20260808 18:27:00.635782 14650 consensus_queue.cc:237] T 00000000000000000000000000000000 P 305a0c62ced7456dadfbc6746f0b809b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "305a0c62ced7456dadfbc6746f0b809b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37945 } }
I20260808 18:27:00.636253 14638 sys_catalog.cc:564] T 00000000000000000000000000000000 P 305a0c62ced7456dadfbc6746f0b809b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:27:00.637451 14651 sys_catalog.cc:455] T 00000000000000000000000000000000 P 305a0c62ced7456dadfbc6746f0b809b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "305a0c62ced7456dadfbc6746f0b809b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "305a0c62ced7456dadfbc6746f0b809b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37945 } } }
I20260808 18:27:00.637578 14651 sys_catalog.cc:458] T 00000000000000000000000000000000 P 305a0c62ced7456dadfbc6746f0b809b [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:00.637864 14652 sys_catalog.cc:455] T 00000000000000000000000000000000 P 305a0c62ced7456dadfbc6746f0b809b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 305a0c62ced7456dadfbc6746f0b809b. Latest consensus state: current_term: 1 leader_uuid: "305a0c62ced7456dadfbc6746f0b809b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "305a0c62ced7456dadfbc6746f0b809b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37945 } } }
I20260808 18:27:00.637971 14652 sys_catalog.cc:458] T 00000000000000000000000000000000 P 305a0c62ced7456dadfbc6746f0b809b [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:00.638265 14660 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:27:00.641247 14660 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:27:00.646440 14660 catalog_manager.cc:1347] Generated new cluster ID: 34f80f5d8f0a4f4baecafc2c4e389018
I20260808 18:27:00.646502 14660 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:27:00.660506 14660 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:27:00.660799 14660 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:27:00.665794 14660 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 305a0c62ced7456dadfbc6746f0b809b: Generated new TSK 0
I20260808 18:27:00.666379 14660 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:27:00.672945 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 14567
2026-08-08T18:27:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1430 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213345431840-23842-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:27:00 dist-test-slave-5k54 krb5kdc[14671](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:27:00 dist-test-slave-5k54 krb5kdc[14671](info): set up 2 sockets
Aug 08 18:27:00 dist-test-slave-5k54 krb5kdc[14671](info): commencing operation
krb5kdc: starting...
W20260808 18:27:01.739876 23842 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Aug 08 18:27:01 dist-test-slave-5k54 krb5kdc[14671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:27:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:27:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.72.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
I20260808 18:27:01.871141 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:32775
--webserver_interface=127.23.72.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:46021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:32775
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:01.980954 14685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:01.981232 14685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:01.981319 14685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:01.985416 14685 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:27:01.985496 14685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:01.985532 14685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:01.985585 14685 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:27:01.985623 14685 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:27:01.990780 14685 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:46021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:32775
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:32775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.72.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:01.991864 14685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:01.992858 14685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:02.001463 14690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:02.001458 14693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:02.001686 14685 server_base.cc:1034] running on GCE node
W20260808 18:27:02.001857 14691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:02.002409 14685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:02.003528 14685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:02.004719 14685 hybrid_clock.cc:648] HybridClock initialized: now 1786213622004698 us; error 30 us; skew 500 ppm
Aug 08 18:27:02 dist-test-slave-5k54 krb5kdc[14671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213622, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:02.007587 14685 init.cc:377] Logged in from keytab as kudu/127.23.72.190@KRBTEST.COM (short username kudu)
I20260808 18:27:02.008462 14685 webserver.cc:460] Webserver started at http://127.23.72.190:33241/ using document root <none> and password file <none>
I20260808 18:27:02.009014 14685 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:02.009078 14685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:02.009325 14685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:02.011382 14685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "5986098d250043258ed72ea6ad3e5413"
format_stamp: "Formatted at 2026-08-08 18:27:02 on dist-test-slave-5k54"
server_key: "82a67223e947d77ba26264e749552419"
server_key_iv: "f1289c485af884c9c0a19297d1d660b9"
server_key_version: "encryptionkey@0"
I20260808 18:27:02.012025 14685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "5986098d250043258ed72ea6ad3e5413"
format_stamp: "Formatted at 2026-08-08 18:27:02 on dist-test-slave-5k54"
server_key: "82a67223e947d77ba26264e749552419"
server_key_iv: "f1289c485af884c9c0a19297d1d660b9"
server_key_version: "encryptionkey@0"
I20260808 18:27:02.015651 14685 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:27:02.017936 14700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:02.019177 14685 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:27:02.019351 14685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "5986098d250043258ed72ea6ad3e5413"
format_stamp: "Formatted at 2026-08-08 18:27:02 on dist-test-slave-5k54"
server_key: "82a67223e947d77ba26264e749552419"
server_key_iv: "f1289c485af884c9c0a19297d1d660b9"
server_key_version: "encryptionkey@0"
I20260808 18:27:02.019503 14685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:02.046326 14685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:02.049278 14685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:02.049449 14685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:02.056634 14685 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.190:32775
I20260808 18:27:02.056666 14757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.190:32775 every 8 connection(s)
I20260808 18:27:02.057603 14685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:27:02.058101 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 14685
I20260808 18:27:02.058267 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:27:02.058583 23842 external_mini_cluster.cc:1428] Setting key a88c5809c36dfd5188484ecd637f0e33
I20260808 18:27:02.060927 14759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:02.066547 14759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5986098d250043258ed72ea6ad3e5413: Bootstrap starting.
Aug 08 18:27:02 dist-test-slave-5k54 krb5kdc[14671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:27:02.069315 14759 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5986098d250043258ed72ea6ad3e5413: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:02.070070 14759 log.cc:824] T 00000000000000000000000000000000 P 5986098d250043258ed72ea6ad3e5413: Log is configured to *not* fsync() on all Append() calls
I20260808 18:27:02.072161 14759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5986098d250043258ed72ea6ad3e5413: No bootstrap required, opened a new log
I20260808 18:27:02.073264 14762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:02.060052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33969 (local address 127.23.72.190:32775)
0808 18:27:02.060486 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:02.060501 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:02.060542 (+ 41us) server_negotiation.cc:407] Connection header received
0808 18:27:02.061299 (+ 757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:02.061318 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:02.061659 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:02.062010 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:02.062833 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:02.063924 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:02.064804 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:02.065282 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:02.065459 (+ 177us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:02.068403 (+ 2944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:02.068408 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:02.068420 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:02.068447 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:02.070196 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:02.070795 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:02.070808 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:02.070812 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:02.070924 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:02.071273 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:02.071283 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:02.071285 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:02.071778 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:02.072000 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:02.072329 (+ 329us) server_negotiation.cc:299] Negotiation successful
0808 18:27:02.072558 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":134,"threads_started":1}
I20260808 18:27:02.076472 23842 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:27:02.076414 14759 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5986098d250043258ed72ea6ad3e5413 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5986098d250043258ed72ea6ad3e5413" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 32775 } }
I20260808 18:27:02.076704 14759 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5986098d250043258ed72ea6ad3e5413 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:02.076764 14759 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5986098d250043258ed72ea6ad3e5413 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5986098d250043258ed72ea6ad3e5413, State: Initialized, Role: FOLLOWER
I20260808 18:27:02.077153 14759 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5986098d250043258ed72ea6ad3e5413 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5986098d250043258ed72ea6ad3e5413" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 32775 } }
I20260808 18:27:02.077306 14759 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5986098d250043258ed72ea6ad3e5413 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:27:02.077402 14759 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5986098d250043258ed72ea6ad3e5413 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:27:02.077507 14759 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5986098d250043258ed72ea6ad3e5413 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:02.078521 14759 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5986098d250043258ed72ea6ad3e5413 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5986098d250043258ed72ea6ad3e5413" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 32775 } }
I20260808 18:27:02.078898 14759 leader_election.cc:302] T 00000000000000000000000000000000 P 5986098d250043258ed72ea6ad3e5413 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5986098d250043258ed72ea6ad3e5413; no voters:
I20260808 18:27:02.079260 14759 leader_election.cc:288] T 00000000000000000000000000000000 P 5986098d250043258ed72ea6ad3e5413 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:27:02.079329 14769 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5986098d250043258ed72ea6ad3e5413 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:27:02.079627 14769 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5986098d250043258ed72ea6ad3e5413 [term 1 LEADER]: Becoming Leader. State: Replica: 5986098d250043258ed72ea6ad3e5413, State: Running, Role: LEADER
I20260808 18:27:02.079972 14769 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5986098d250043258ed72ea6ad3e5413 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5986098d250043258ed72ea6ad3e5413" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 32775 } }
I20260808 18:27:02.080632 14759 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5986098d250043258ed72ea6ad3e5413 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:27:02.081614 14770 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5986098d250043258ed72ea6ad3e5413 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5986098d250043258ed72ea6ad3e5413" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5986098d250043258ed72ea6ad3e5413" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 32775 } } }
I20260808 18:27:02.081733 14770 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5986098d250043258ed72ea6ad3e5413 [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:02.082037 14771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5986098d250043258ed72ea6ad3e5413 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5986098d250043258ed72ea6ad3e5413. Latest consensus state: current_term: 1 leader_uuid: "5986098d250043258ed72ea6ad3e5413" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5986098d250043258ed72ea6ad3e5413" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 32775 } } }
I20260808 18:27:02.082156 14771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5986098d250043258ed72ea6ad3e5413 [sys.catalog]: This master's current role is: LEADER
W20260808 18:27:02.085711 14786 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5986098d250043258ed72ea6ad3e5413: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:27:02.085793 14786 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:27:02.085872 14779 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:27:02.086695 14779 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:27:02.087797 14762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:02.078729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49175 (local address 127.23.72.190:32775)
0808 18:27:02.078859 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:02.078863 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:02.079054 (+ 191us) server_negotiation.cc:407] Connection header received
0808 18:27:02.079172 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:02.079174 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:02.079211 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:02.079299 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:02.080107 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:02.080699 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:02.081906 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:02.082292 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:02.082415 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:02.084360 (+ 1945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:02.084364 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:02.084366 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:02.084390 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:02.086018 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:02.086482 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:02.086485 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:02.086487 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:02.086572 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:02.086905 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:02.086908 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:02.086910 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:02.087086 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:02.087237 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:02.087535 (+ 298us) server_negotiation.cc:299] Negotiation successful
0808 18:27:02.087627 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 18:27:02.092093 14779 catalog_manager.cc:1347] Generated new cluster ID: 751a43bde9fc4ea08f01bf7936526f67
I20260808 18:27:02.092206 14779 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:27:02.104777 14779 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:27:02.106006 14779 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:27:02.112874 14779 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5986098d250043258ed72ea6ad3e5413: Generated new TSK 0
I20260808 18:27:02.113678 14779 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:27:02.124963 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 14685
W20260808 18:27:02.132958 14669 connection.cc:535] client connection to 127.23.72.190:32775 recv error: Network error: failed to read from TLS socket (remote: 127.23.72.190:32775): Connection reset by peer (error 104)
2026-08-08T18:27:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1451 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213345431840-23842-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:27:02 dist-test-slave-5k54 krb5kdc[14793](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:27:02 dist-test-slave-5k54 krb5kdc[14793](info): set up 2 sockets
Aug 08 18:27:02 dist-test-slave-5k54 krb5kdc[14793](info): commencing operation
krb5kdc: starting...
W20260808 18:27:03.196149 23842 mini_kdc.cc:121] Time spent starting KDC: real 1.049s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Aug 08 18:27:03 dist-test-slave-5k54 krb5kdc[14793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213623, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:27:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:27:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.72.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
I20260808 18:27:03.323274 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:34671
--webserver_interface=127.23.72.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:36837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:34671
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:03.433152 14807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:03.433475 14807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:03.433555 14807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:03.437625 14807 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:27:03.437717 14807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:03.437754 14807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:03.437793 14807 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:27:03.437825 14807 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:27:03.443068 14807 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:36837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:34671
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:34671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.72.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14807
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:03.444146 14807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:03.445187 14807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:03.453619 14814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:03.453740 14807 server_base.cc:1034] running on GCE node
W20260808 18:27:03.453574 14817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:03.453585 14815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:03.454361 14807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:03.455480 14807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:03.456696 14807 hybrid_clock.cc:648] HybridClock initialized: now 1786213623456676 us; error 30 us; skew 500 ppm
Aug 08 18:27:03 dist-test-slave-5k54 krb5kdc[14793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213623, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:03.459571 14807 init.cc:377] Logged in from keytab as kudu/127.23.72.190@KRBTEST.COM (short username kudu)
I20260808 18:27:03.460403 14807 webserver.cc:460] Webserver started at http://127.23.72.190:46851/ using document root <none> and password file <none>
I20260808 18:27:03.460949 14807 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:03.461038 14807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:03.461269 14807 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:03.463240 14807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "80c83fb7bd69439fb4bd06e9d7a0d05a"
format_stamp: "Formatted at 2026-08-08 18:27:03 on dist-test-slave-5k54"
server_key: "84542ac261b3b09f8f25d8df9f4550c7"
server_key_iv: "e2800aa08e3c7d2aa9bbb834ac386a81"
server_key_version: "encryptionkey@0"
I20260808 18:27:03.463860 14807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "80c83fb7bd69439fb4bd06e9d7a0d05a"
format_stamp: "Formatted at 2026-08-08 18:27:03 on dist-test-slave-5k54"
server_key: "84542ac261b3b09f8f25d8df9f4550c7"
server_key_iv: "e2800aa08e3c7d2aa9bbb834ac386a81"
server_key_version: "encryptionkey@0"
I20260808 18:27:03.467548 14807 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:27:03.469812 14824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:03.471042 14807 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:27:03.471172 14807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "80c83fb7bd69439fb4bd06e9d7a0d05a"
format_stamp: "Formatted at 2026-08-08 18:27:03 on dist-test-slave-5k54"
server_key: "84542ac261b3b09f8f25d8df9f4550c7"
server_key_iv: "e2800aa08e3c7d2aa9bbb834ac386a81"
server_key_version: "encryptionkey@0"
I20260808 18:27:03.471271 14807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:03.481393 14807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:03.484412 14807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:03.484611 14807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:03.491849 14807 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.190:34671
I20260808 18:27:03.491875 14878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.190:34671 every 8 connection(s)
I20260808 18:27:03.492805 14807 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:27:03.495548 14879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:03.499861 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 14807
I20260808 18:27:03.500007 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:27:03.500311 23842 external_mini_cluster.cc:1428] Setting key ae7e00e84b999ab5a50ff2f5b56f7aed
I20260808 18:27:03.501422 14879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80c83fb7bd69439fb4bd06e9d7a0d05a: Bootstrap starting.
I20260808 18:27:03.503717 14879 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 80c83fb7bd69439fb4bd06e9d7a0d05a: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:03.504468 14879 log.cc:824] T 00000000000000000000000000000000 P 80c83fb7bd69439fb4bd06e9d7a0d05a: Log is configured to *not* fsync() on all Append() calls
I20260808 18:27:03.506309 14879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80c83fb7bd69439fb4bd06e9d7a0d05a: No bootstrap required, opened a new log
I20260808 18:27:03.509382 14882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:03.501748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53673 (local address 127.23.72.190:34671)
0808 18:27:03.502173 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:03.502190 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:03.502227 (+ 37us) server_negotiation.cc:407] Connection header received
0808 18:27:03.502887 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:03.502903 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:03.503229 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:03.503571 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:03.503969 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:03.505027 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:03.505961 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:03.506511 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:03.506682 (+ 171us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:03.507676 (+ 994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:03.507681 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:03.507692 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:27:03.508122 (+ 430us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:03.508178 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:27:03.508200 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:03.508350 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:03.508550 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 18:27:03.508686 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":128,"threads_started":1}
I20260808 18:27:03.510018 14879 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80c83fb7bd69439fb4bd06e9d7a0d05a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80c83fb7bd69439fb4bd06e9d7a0d05a" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 34671 } }
I20260808 18:27:03.510224 14879 raft_consensus.cc:374] T 00000000000000000000000000000000 P 80c83fb7bd69439fb4bd06e9d7a0d05a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:03.510281 14879 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80c83fb7bd69439fb4bd06e9d7a0d05a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80c83fb7bd69439fb4bd06e9d7a0d05a, State: Initialized, Role: FOLLOWER
I20260808 18:27:03.510705 14879 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80c83fb7bd69439fb4bd06e9d7a0d05a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80c83fb7bd69439fb4bd06e9d7a0d05a" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 34671 } }
I20260808 18:27:03.510854 14879 raft_consensus.cc:388] T 00000000000000000000000000000000 P 80c83fb7bd69439fb4bd06e9d7a0d05a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:27:03.510941 14879 raft_consensus.cc:482] T 00000000000000000000000000000000 P 80c83fb7bd69439fb4bd06e9d7a0d05a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:27:03.511063 14879 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80c83fb7bd69439fb4bd06e9d7a0d05a [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:03.512166 14879 raft_consensus.cc:504] T 00000000000000000000000000000000 P 80c83fb7bd69439fb4bd06e9d7a0d05a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80c83fb7bd69439fb4bd06e9d7a0d05a" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 34671 } }
I20260808 18:27:03.512509 23842 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:27:03.512549 14879 leader_election.cc:302] T 00000000000000000000000000000000 P 80c83fb7bd69439fb4bd06e9d7a0d05a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 80c83fb7bd69439fb4bd06e9d7a0d05a; no voters:
I20260808 18:27:03.512919 14879 leader_election.cc:288] T 00000000000000000000000000000000 P 80c83fb7bd69439fb4bd06e9d7a0d05a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:27:03.512988 14884 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 80c83fb7bd69439fb4bd06e9d7a0d05a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:27:03.513190 14884 raft_consensus.cc:686] T 00000000000000000000000000000000 P 80c83fb7bd69439fb4bd06e9d7a0d05a [term 1 LEADER]: Becoming Leader. State: Replica: 80c83fb7bd69439fb4bd06e9d7a0d05a, State: Running, Role: LEADER
I20260808 18:27:03.513545 14884 consensus_queue.cc:237] T 00000000000000000000000000000000 P 80c83fb7bd69439fb4bd06e9d7a0d05a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80c83fb7bd69439fb4bd06e9d7a0d05a" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 34671 } }
I20260808 18:27:03.514164 14879 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80c83fb7bd69439fb4bd06e9d7a0d05a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:27:03.515414 14886 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80c83fb7bd69439fb4bd06e9d7a0d05a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "80c83fb7bd69439fb4bd06e9d7a0d05a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80c83fb7bd69439fb4bd06e9d7a0d05a" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 34671 } } }
I20260808 18:27:03.515569 14886 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80c83fb7bd69439fb4bd06e9d7a0d05a [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:03.515858 14885 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80c83fb7bd69439fb4bd06e9d7a0d05a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80c83fb7bd69439fb4bd06e9d7a0d05a. Latest consensus state: current_term: 1 leader_uuid: "80c83fb7bd69439fb4bd06e9d7a0d05a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80c83fb7bd69439fb4bd06e9d7a0d05a" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 34671 } } }
I20260808 18:27:03.515952 14885 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80c83fb7bd69439fb4bd06e9d7a0d05a [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:03.515987 14899 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:27:03.519172 14899 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:27:03.520198 14882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:03.514775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53057 (local address 127.23.72.190:34671)
0808 18:27:03.514903 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:03.514907 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:03.515325 (+ 418us) server_negotiation.cc:407] Connection header received
0808 18:27:03.515540 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:03.515543 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:03.515594 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:03.515665 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:03.516119 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:03.516848 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:03.518032 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:03.518356 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:03.518435 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:03.519532 (+ 1097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:03.519535 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:03.519538 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:27:03.519551 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:03.519580 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:27:03.519594 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:03.519686 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:03.520041 (+ 355us) server_negotiation.cc:299] Negotiation successful
0808 18:27:03.520076 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 18:27:03.525272 14899 catalog_manager.cc:1347] Generated new cluster ID: fb3194b2e7964a63bbf73a84a9fac77a
I20260808 18:27:03.525336 14899 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:27:03.532342 14899 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:27:03.533255 14899 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:27:03.539518 14899 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 80c83fb7bd69439fb4bd06e9d7a0d05a: Generated new TSK 0
I20260808 18:27:03.540169 14899 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:27:03.548029 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 14807
W20260808 18:27:03.556520 14791 connection.cc:535] client connection to 127.23.72.190:34671 recv error: Network error: failed to read from TLS socket (remote: 127.23.72.190:34671): Connection reset by peer (error 104)
2026-08-08T18:27:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1423 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213345431840-23842-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:27:03 dist-test-slave-5k54 krb5kdc[14912](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:27:03 dist-test-slave-5k54 krb5kdc[14912](info): set up 2 sockets
Aug 08 18:27:03 dist-test-slave-5k54 krb5kdc[14912](info): commencing operation
krb5kdc: starting...
W20260808 18:27:04.620419 23842 mini_kdc.cc:121] Time spent starting KDC: real 1.049s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Aug 08 18:27:04 dist-test-slave-5k54 krb5kdc[14912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:27:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:27:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.72.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
I20260808 18:27:04.748591 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:43451
--webserver_interface=127.23.72.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:46105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:43451
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:04.861043 14929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:04.861312 14929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:04.861353 14929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:04.865449 14929 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:27:04.865506 14929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:04.865526 14929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:04.865545 14929 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:27:04.865561 14929 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:27:04.872345 14929 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:46105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:43451
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:43451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.72.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14929
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:04.873746 14929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:04.875077 14929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:04.883633 14937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:04.883682 14929 server_base.cc:1034] running on GCE node
W20260808 18:27:04.883641 14935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:04.883646 14934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:04.884569 14929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:04.885643 14929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:04.886824 14929 hybrid_clock.cc:648] HybridClock initialized: now 1786213624886799 us; error 42 us; skew 500 ppm
Aug 08 18:27:04 dist-test-slave-5k54 krb5kdc[14912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213624, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:04.889624 14929 init.cc:377] Logged in from keytab as kudu/127.23.72.190@KRBTEST.COM (short username kudu)
I20260808 18:27:04.890538 14929 webserver.cc:460] Webserver started at http://127.23.72.190:42331/ using document root <none> and password file <none>
I20260808 18:27:04.891091 14929 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:04.891179 14929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:04.891472 14929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:04.893532 14929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "04c6d9a2bdf448f9981edb203627713f"
format_stamp: "Formatted at 2026-08-08 18:27:04 on dist-test-slave-5k54"
server_key: "436510f1532139ada7115277b88ae874"
server_key_iv: "fa062948ff533ec67d22413f38f4529a"
server_key_version: "encryptionkey@0"
I20260808 18:27:04.894100 14929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "04c6d9a2bdf448f9981edb203627713f"
format_stamp: "Formatted at 2026-08-08 18:27:04 on dist-test-slave-5k54"
server_key: "436510f1532139ada7115277b88ae874"
server_key_iv: "fa062948ff533ec67d22413f38f4529a"
server_key_version: "encryptionkey@0"
I20260808 18:27:04.897746 14929 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260808 18:27:04.900167 14944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:04.901356 14929 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 18:27:04.901497 14929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "04c6d9a2bdf448f9981edb203627713f"
format_stamp: "Formatted at 2026-08-08 18:27:04 on dist-test-slave-5k54"
server_key: "436510f1532139ada7115277b88ae874"
server_key_iv: "fa062948ff533ec67d22413f38f4529a"
server_key_version: "encryptionkey@0"
I20260808 18:27:04.901604 14929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:04.906188 14929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:04.909138 14929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:04.909329 14929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:04.916299 14929 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.190:43451
I20260808 18:27:04.916333 15000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.190:43451 every 8 connection(s)
I20260808 18:27:04.917203 14929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:27:04.919934 15001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:04.925028 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 14929
I20260808 18:27:04.925191 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:27:04.925482 15001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04c6d9a2bdf448f9981edb203627713f: Bootstrap starting.
I20260808 18:27:04.925518 23842 external_mini_cluster.cc:1428] Setting key 694f3adb790b13878d3b785d92a0c25e
I20260808 18:27:04.928009 15001 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04c6d9a2bdf448f9981edb203627713f: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:04.928759 15001 log.cc:824] T 00000000000000000000000000000000 P 04c6d9a2bdf448f9981edb203627713f: Log is configured to *not* fsync() on all Append() calls
I20260808 18:27:04.930652 15005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:04.926847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50225 (local address 127.23.72.190:43451)
0808 18:27:04.927271 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:04.927285 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:04.927320 (+ 35us) server_negotiation.cc:407] Connection header received
0808 18:27:04.927991 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:04.928007 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:04.928345 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:04.928671 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:04.929395 (+ 724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:04.929403 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:04.929421 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:27:04.929433 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:04.929513 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:27:04.929547 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:04.929664 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:04.929822 (+ 158us) server_negotiation.cc:299] Negotiation successful
0808 18:27:04.929984 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":127,"threads_started":1}
I20260808 18:27:04.931098 15001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04c6d9a2bdf448f9981edb203627713f: No bootstrap required, opened a new log
I20260808 18:27:04.933552 23842 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:27:04.935755 15001 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04c6d9a2bdf448f9981edb203627713f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04c6d9a2bdf448f9981edb203627713f" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 43451 } }
I20260808 18:27:04.936112 15001 raft_consensus.cc:374] T 00000000000000000000000000000000 P 04c6d9a2bdf448f9981edb203627713f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:04.936197 15001 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04c6d9a2bdf448f9981edb203627713f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04c6d9a2bdf448f9981edb203627713f, State: Initialized, Role: FOLLOWER
I20260808 18:27:04.936738 15001 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04c6d9a2bdf448f9981edb203627713f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04c6d9a2bdf448f9981edb203627713f" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 43451 } }
I20260808 18:27:04.936920 15001 raft_consensus.cc:388] T 00000000000000000000000000000000 P 04c6d9a2bdf448f9981edb203627713f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:27:04.936992 15001 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04c6d9a2bdf448f9981edb203627713f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:27:04.937101 15001 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04c6d9a2bdf448f9981edb203627713f [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:04.937666 15005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:04.935883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60701 (local address 127.23.72.190:43451)
0808 18:27:04.936024 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:04.936029 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:04.936223 (+ 194us) server_negotiation.cc:407] Connection header received
0808 18:27:04.936358 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:04.936360 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:04.936388 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:04.936459 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:04.937290 (+ 831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:04.937293 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:04.937296 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:27:04.937307 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:04.937340 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:27:04.937350 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:04.937441 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:04.937517 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 18:27:04.937555 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 18:27:04.938231 15001 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04c6d9a2bdf448f9981edb203627713f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04c6d9a2bdf448f9981edb203627713f" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 43451 } }
I20260808 18:27:04.938601 15001 leader_election.cc:302] T 00000000000000000000000000000000 P 04c6d9a2bdf448f9981edb203627713f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 04c6d9a2bdf448f9981edb203627713f; no voters:
I20260808 18:27:04.938926 15001 leader_election.cc:288] T 00000000000000000000000000000000 P 04c6d9a2bdf448f9981edb203627713f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:27:04.939034 15012 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04c6d9a2bdf448f9981edb203627713f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:27:04.939281 15012 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04c6d9a2bdf448f9981edb203627713f [term 1 LEADER]: Becoming Leader. State: Replica: 04c6d9a2bdf448f9981edb203627713f, State: Running, Role: LEADER
I20260808 18:27:04.939642 15012 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04c6d9a2bdf448f9981edb203627713f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04c6d9a2bdf448f9981edb203627713f" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 43451 } }
I20260808 18:27:04.940166 15001 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04c6d9a2bdf448f9981edb203627713f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:27:04.941187 15014 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04c6d9a2bdf448f9981edb203627713f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04c6d9a2bdf448f9981edb203627713f. Latest consensus state: current_term: 1 leader_uuid: "04c6d9a2bdf448f9981edb203627713f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04c6d9a2bdf448f9981edb203627713f" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 43451 } } }
I20260808 18:27:04.941315 15014 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04c6d9a2bdf448f9981edb203627713f [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:04.941746 15021 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:27:04.942073 15013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04c6d9a2bdf448f9981edb203627713f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "04c6d9a2bdf448f9981edb203627713f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04c6d9a2bdf448f9981edb203627713f" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 43451 } } }
I20260808 18:27:04.942219 15013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04c6d9a2bdf448f9981edb203627713f [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:04.944976 15021 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:27:04.950392 15021 catalog_manager.cc:1347] Generated new cluster ID: 19166049124149598fe24ca895fa7048
I20260808 18:27:04.950462 15021 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:27:04.959975 15021 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:27:04.960275 15021 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:27:04.965137 15021 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 04c6d9a2bdf448f9981edb203627713f: Generated new TSK 0
I20260808 18:27:04.965837 15021 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:27:04.974475 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 14929
2026-08-08T18:27:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1424 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213345431840-23842-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:27:05 dist-test-slave-5k54 krb5kdc[15036](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:27:05 dist-test-slave-5k54 krb5kdc[15036](info): set up 2 sockets
Aug 08 18:27:05 dist-test-slave-5k54 krb5kdc[15036](info): commencing operation
krb5kdc: starting...
W20260808 18:27:06.064551 23842 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Aug 08 18:27:06 dist-test-slave-5k54 krb5kdc[15036](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:27:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:27:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.72.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
I20260808 18:27:06.189807 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:40425
--webserver_interface=127.23.72.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:33477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:40425
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:06.298564 15052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:06.298825 15052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:06.298866 15052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:06.302819 15052 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:27:06.302875 15052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:06.302894 15052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:06.302910 15052 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:27:06.302958 15052 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:27:06.308095 15052 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:33477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:40425
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:40425
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.72.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15052
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:06.309091 15052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:06.310092 15052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:06.318389 15057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:06.318323 15060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:06.318342 15058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:06.318495 15052 server_base.cc:1034] running on GCE node
I20260808 18:27:06.319049 15052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:06.320116 15052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:06.321295 15052 hybrid_clock.cc:648] HybridClock initialized: now 1786213626321266 us; error 40 us; skew 500 ppm
Aug 08 18:27:06 dist-test-slave-5k54 krb5kdc[15036](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213626, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:06.329736 15052 init.cc:377] Logged in from keytab as kudu/127.23.72.190@KRBTEST.COM (short username kudu)
I20260808 18:27:06.330672 15052 webserver.cc:460] Webserver started at http://127.23.72.190:39911/ using document root <none> and password file <none>
I20260808 18:27:06.331236 15052 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:06.331326 15052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:06.331597 15052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:06.333642 15052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "18b4de2327ae4cc8a639233f0d9698c8"
format_stamp: "Formatted at 2026-08-08 18:27:06 on dist-test-slave-5k54"
server_key: "d44f8f34219166663751d979c25d8d3a"
server_key_iv: "7d5f7e68d66438bcf4503189dd354ef2"
server_key_version: "encryptionkey@0"
I20260808 18:27:06.334210 15052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "18b4de2327ae4cc8a639233f0d9698c8"
format_stamp: "Formatted at 2026-08-08 18:27:06 on dist-test-slave-5k54"
server_key: "d44f8f34219166663751d979c25d8d3a"
server_key_iv: "7d5f7e68d66438bcf4503189dd354ef2"
server_key_version: "encryptionkey@0"
I20260808 18:27:06.337980 15052 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 18:27:06.340399 15069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:06.341606 15052 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:27:06.341743 15052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "18b4de2327ae4cc8a639233f0d9698c8"
format_stamp: "Formatted at 2026-08-08 18:27:06 on dist-test-slave-5k54"
server_key: "d44f8f34219166663751d979c25d8d3a"
server_key_iv: "7d5f7e68d66438bcf4503189dd354ef2"
server_key_version: "encryptionkey@0"
I20260808 18:27:06.341888 15052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:06.368173 15052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:06.370976 15052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:06.371177 15052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:06.378451 15052 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.190:40425
I20260808 18:27:06.378475 15122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.190:40425 every 8 connection(s)
I20260808 18:27:06.379361 15052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:27:06.382055 15123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:06.386044 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 15052
I20260808 18:27:06.386199 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:27:06.386521 23842 external_mini_cluster.cc:1428] Setting key fe65a51e0bbb4c4c1d7bf353e877a710
I20260808 18:27:06.387826 15123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18b4de2327ae4cc8a639233f0d9698c8: Bootstrap starting.
I20260808 18:27:06.390074 15123 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 18b4de2327ae4cc8a639233f0d9698c8: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:06.390838 15123 log.cc:824] T 00000000000000000000000000000000 P 18b4de2327ae4cc8a639233f0d9698c8: Log is configured to *not* fsync() on all Append() calls
I20260808 18:27:06.392776 15123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18b4de2327ae4cc8a639233f0d9698c8: No bootstrap required, opened a new log
Aug 08 18:27:06 dist-test-slave-5k54 krb5kdc[15036](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:27:06.396533 15123 raft_consensus.cc:348] T 00000000000000000000000000000000 P 18b4de2327ae4cc8a639233f0d9698c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18b4de2327ae4cc8a639233f0d9698c8" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40425 } }
I20260808 18:27:06.396797 15123 raft_consensus.cc:374] T 00000000000000000000000000000000 P 18b4de2327ae4cc8a639233f0d9698c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:06.396907 15123 raft_consensus.cc:729] T 00000000000000000000000000000000 P 18b4de2327ae4cc8a639233f0d9698c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18b4de2327ae4cc8a639233f0d9698c8, State: Initialized, Role: FOLLOWER
I20260808 18:27:06.397428 15123 consensus_queue.cc:260] T 00000000000000000000000000000000 P 18b4de2327ae4cc8a639233f0d9698c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18b4de2327ae4cc8a639233f0d9698c8" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40425 } }
I20260808 18:27:06.397589 15123 raft_consensus.cc:388] T 00000000000000000000000000000000 P 18b4de2327ae4cc8a639233f0d9698c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:27:06.397677 15123 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18b4de2327ae4cc8a639233f0d9698c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:27:06.397797 15123 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18b4de2327ae4cc8a639233f0d9698c8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:06.398681 15123 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18b4de2327ae4cc8a639233f0d9698c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18b4de2327ae4cc8a639233f0d9698c8" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40425 } }
I20260808 18:27:06.399024 15123 leader_election.cc:302] T 00000000000000000000000000000000 P 18b4de2327ae4cc8a639233f0d9698c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 18b4de2327ae4cc8a639233f0d9698c8; no voters:
I20260808 18:27:06.399328 15123 leader_election.cc:288] T 00000000000000000000000000000000 P 18b4de2327ae4cc8a639233f0d9698c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:27:06.399447 15130 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 18b4de2327ae4cc8a639233f0d9698c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:27:06.399693 15130 raft_consensus.cc:686] T 00000000000000000000000000000000 P 18b4de2327ae4cc8a639233f0d9698c8 [term 1 LEADER]: Becoming Leader. State: Replica: 18b4de2327ae4cc8a639233f0d9698c8, State: Running, Role: LEADER
I20260808 18:27:06.400089 15130 consensus_queue.cc:237] T 00000000000000000000000000000000 P 18b4de2327ae4cc8a639233f0d9698c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18b4de2327ae4cc8a639233f0d9698c8" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40425 } }
I20260808 18:27:06.400650 15123 sys_catalog.cc:564] T 00000000000000000000000000000000 P 18b4de2327ae4cc8a639233f0d9698c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:27:06.401279 15130 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18b4de2327ae4cc8a639233f0d9698c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18b4de2327ae4cc8a639233f0d9698c8. Latest consensus state: current_term: 1 leader_uuid: "18b4de2327ae4cc8a639233f0d9698c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18b4de2327ae4cc8a639233f0d9698c8" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40425 } } }
I20260808 18:27:06.401400 15128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:06.387940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40215 (local address 127.23.72.190:40425)
0808 18:27:06.388349 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:06.388370 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:06.388406 (+ 36us) server_negotiation.cc:407] Connection header received
0808 18:27:06.389070 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:06.389087 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:06.389426 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:06.389755 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:06.390577 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:06.391653 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:06.392572 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:06.393088 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:06.393251 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:06.395879 (+ 2628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:06.395884 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:06.395897 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:06.395923 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:06.397744 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:06.398197 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:06.398203 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:06.398208 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:06.398330 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:06.398617 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:06.398622 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:06.398624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:06.399069 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:06.399265 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:06.399998 (+ 733us) server_negotiation.cc:299] Negotiation successful
0808 18:27:06.400295 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":128,"threads_started":1}
I20260808 18:27:06.401463 15130 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18b4de2327ae4cc8a639233f0d9698c8 [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:06.401715 15131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18b4de2327ae4cc8a639233f0d9698c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "18b4de2327ae4cc8a639233f0d9698c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18b4de2327ae4cc8a639233f0d9698c8" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40425 } } }
I20260808 18:27:06.401803 15131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18b4de2327ae4cc8a639233f0d9698c8 [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:06.402064 15139 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:27:06.404804 23842 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:27:06.405201 15139 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:27:06.412343 15139 catalog_manager.cc:1347] Generated new cluster ID: fa5020b34d244f148357d21db3eb86af
I20260808 18:27:06.412406 15139 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:27:06.418398 15128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:06.407045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54531 (local address 127.23.72.190:40425)
0808 18:27:06.407148 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:06.407152 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:06.407737 (+ 585us) server_negotiation.cc:407] Connection header received
0808 18:27:06.407877 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:06.407880 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:06.407926 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:06.408009 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:06.408646 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:06.409153 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:06.410145 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:06.410410 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:06.410518 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:06.411970 (+ 1452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:06.411973 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:06.411976 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:06.412006 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:06.416717 (+ 4711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:06.417222 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:06.417224 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:06.417226 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:06.417304 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:06.417597 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:06.417599 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:06.417601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:06.417784 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:06.417891 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:06.418139 (+ 248us) server_negotiation.cc:299] Negotiation successful
0808 18:27:06.418246 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 18:27:06.428920 15139 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:27:06.429826 15139 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:27:06.435407 15139 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 18b4de2327ae4cc8a639233f0d9698c8: Generated new TSK 0
I20260808 18:27:06.436121 15139 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:27:06.443137 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 15052
2026-08-08T18:27:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1469 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213345431840-23842-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:27:06 dist-test-slave-5k54 krb5kdc[15158](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:27:06 dist-test-slave-5k54 krb5kdc[15158](info): set up 2 sockets
Aug 08 18:27:06 dist-test-slave-5k54 krb5kdc[15158](info): commencing operation
krb5kdc: starting...
W20260808 18:27:07.532035 23842 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Aug 08 18:27:07 dist-test-slave-5k54 krb5kdc[15158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213627, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:27:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:27:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.72.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
I20260808 18:27:07.660148 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:38719
--webserver_interface=127.23.72.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:34775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:38719
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:07.772588 15174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:07.772867 15174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:07.772943 15174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:07.777076 15174 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:27:07.777132 15174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:07.777151 15174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:07.777168 15174 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:27:07.777182 15174 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:27:07.782735 15174 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:34775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:38719
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:38719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.72.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15174
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:07.784184 15174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:07.785529 15174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:07.794260 15183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:07.794317 15181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:07.794308 15180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:07.794431 15174 server_base.cc:1034] running on GCE node
I20260808 18:27:07.795087 15174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:07.796278 15174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:07.797469 15174 hybrid_clock.cc:648] HybridClock initialized: now 1786213627797437 us; error 41 us; skew 500 ppm
Aug 08 18:27:07 dist-test-slave-5k54 krb5kdc[15158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213627, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:07.800542 15174 init.cc:377] Logged in from keytab as kudu/127.23.72.190@KRBTEST.COM (short username kudu)
I20260808 18:27:07.801533 15174 webserver.cc:460] Webserver started at http://127.23.72.190:38647/ using document root <none> and password file <none>
I20260808 18:27:07.802099 15174 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:07.802162 15174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:07.802418 15174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:07.804625 15174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "6f26b2aa120e4c0094da93e0433eed70"
format_stamp: "Formatted at 2026-08-08 18:27:07 on dist-test-slave-5k54"
server_key: "8c11a47ef59d5e49383440d82e5388a5"
server_key_iv: "329b7eac116535749644d489622bf561"
server_key_version: "encryptionkey@0"
I20260808 18:27:07.805258 15174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "6f26b2aa120e4c0094da93e0433eed70"
format_stamp: "Formatted at 2026-08-08 18:27:07 on dist-test-slave-5k54"
server_key: "8c11a47ef59d5e49383440d82e5388a5"
server_key_iv: "329b7eac116535749644d489622bf561"
server_key_version: "encryptionkey@0"
I20260808 18:27:07.809196 15174 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 18:27:07.811757 15191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:07.813083 15174 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 18:27:07.813242 15174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "6f26b2aa120e4c0094da93e0433eed70"
format_stamp: "Formatted at 2026-08-08 18:27:07 on dist-test-slave-5k54"
server_key: "8c11a47ef59d5e49383440d82e5388a5"
server_key_iv: "329b7eac116535749644d489622bf561"
server_key_version: "encryptionkey@0"
I20260808 18:27:07.813369 15174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:07.826589 15174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:07.829733 15174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:07.829974 15174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:07.837508 15174 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.190:38719
I20260808 18:27:07.837536 15245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.190:38719 every 8 connection(s)
I20260808 18:27:07.838503 15174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:27:07.841212 15246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:07.846812 15246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f26b2aa120e4c0094da93e0433eed70: Bootstrap starting.
I20260808 18:27:07.847186 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 15174
I20260808 18:27:07.847362 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:27:07.847679 23842 external_mini_cluster.cc:1428] Setting key a63b8e54dfb77463121e6af20479a28f
I20260808 18:27:07.849612 15246 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f26b2aa120e4c0094da93e0433eed70: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:07.850528 15246 log.cc:824] T 00000000000000000000000000000000 P 6f26b2aa120e4c0094da93e0433eed70: Log is configured to *not* fsync() on all Append() calls
I20260808 18:27:07.852900 15246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f26b2aa120e4c0094da93e0433eed70: No bootstrap required, opened a new log
I20260808 18:27:07.857596 15246 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f26b2aa120e4c0094da93e0433eed70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f26b2aa120e4c0094da93e0433eed70" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 38719 } }
I20260808 18:27:07.857864 15246 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f26b2aa120e4c0094da93e0433eed70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:07.857913 15246 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f26b2aa120e4c0094da93e0433eed70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f26b2aa120e4c0094da93e0433eed70, State: Initialized, Role: FOLLOWER
I20260808 18:27:07.858158 15249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:07.849153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38965 (local address 127.23.72.190:38719)
0808 18:27:07.849623 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:07.849645 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:07.849691 (+ 46us) server_negotiation.cc:407] Connection header received
0808 18:27:07.850592 (+ 901us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:07.850616 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:07.851073 (+ 457us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:07.851397 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:07.851848 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:07.853283 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:07.854188 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:07.854923 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:07.855088 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:07.856036 (+ 948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:07.856045 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:07.856066 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:27:07.856606 (+ 540us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:07.856673 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:27:07.856704 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:07.856810 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:07.856968 (+ 158us) server_negotiation.cc:299] Negotiation successful
0808 18:27:07.857183 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":128,"threads_started":1}
I20260808 18:27:07.858394 15246 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f26b2aa120e4c0094da93e0433eed70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f26b2aa120e4c0094da93e0433eed70" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 38719 } }
I20260808 18:27:07.858560 15246 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f26b2aa120e4c0094da93e0433eed70 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:27:07.858645 15246 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f26b2aa120e4c0094da93e0433eed70 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:27:07.858743 15246 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f26b2aa120e4c0094da93e0433eed70 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:07.859906 15246 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f26b2aa120e4c0094da93e0433eed70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f26b2aa120e4c0094da93e0433eed70" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 38719 } }
I20260808 18:27:07.860278 15246 leader_election.cc:302] T 00000000000000000000000000000000 P 6f26b2aa120e4c0094da93e0433eed70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f26b2aa120e4c0094da93e0433eed70; no voters:
I20260808 18:27:07.860641 15246 leader_election.cc:288] T 00000000000000000000000000000000 P 6f26b2aa120e4c0094da93e0433eed70 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:27:07.860759 15251 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f26b2aa120e4c0094da93e0433eed70 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:27:07.860994 15251 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f26b2aa120e4c0094da93e0433eed70 [term 1 LEADER]: Becoming Leader. State: Replica: 6f26b2aa120e4c0094da93e0433eed70, State: Running, Role: LEADER
I20260808 18:27:07.861323 23842 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:27:07.861327 15251 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f26b2aa120e4c0094da93e0433eed70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f26b2aa120e4c0094da93e0433eed70" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 38719 } }
I20260808 18:27:07.861920 15246 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f26b2aa120e4c0094da93e0433eed70 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:27:07.863037 15252 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f26b2aa120e4c0094da93e0433eed70 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f26b2aa120e4c0094da93e0433eed70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f26b2aa120e4c0094da93e0433eed70" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 38719 } } }
I20260808 18:27:07.863171 15252 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f26b2aa120e4c0094da93e0433eed70 [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:07.863487 15253 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f26b2aa120e4c0094da93e0433eed70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f26b2aa120e4c0094da93e0433eed70. Latest consensus state: current_term: 1 leader_uuid: "6f26b2aa120e4c0094da93e0433eed70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f26b2aa120e4c0094da93e0433eed70" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 38719 } } }
I20260808 18:27:07.863580 15253 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f26b2aa120e4c0094da93e0433eed70 [sys.catalog]: This master's current role is: LEADER
W20260808 18:27:07.867398 15275 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6f26b2aa120e4c0094da93e0433eed70: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:27:07.867506 15275 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:27:07.867592 15267 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:27:07.868007 15249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:07.863610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56717 (local address 127.23.72.190:38719)
0808 18:27:07.863707 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:07.863711 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:07.864002 (+ 291us) server_negotiation.cc:407] Connection header received
0808 18:27:07.864173 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:07.864180 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:07.864240 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:07.864325 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:07.864782 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:07.865367 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:07.866362 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:07.866601 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:07.866708 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:07.867527 (+ 819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:07.867534 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:07.867538 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:27:07.867551 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:07.867581 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:27:07.867596 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:07.867659 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:07.867806 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 18:27:07.867855 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 18:27:07.868639 15267 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:27:07.873930 15267 catalog_manager.cc:1347] Generated new cluster ID: 158a1c6e064348eba4c83eac228301cd
I20260808 18:27:07.873996 15267 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:27:07.887368 15267 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:27:07.888284 15267 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:27:07.901669 15267 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f26b2aa120e4c0094da93e0433eed70: Generated new TSK 0
I20260808 18:27:07.902385 15267 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:27:07.912837 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 15174
2026-08-08T18:27:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1468 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213345431840-23842-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:27:07 dist-test-slave-5k54 krb5kdc[15282](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:27:07 dist-test-slave-5k54 krb5kdc[15282](info): set up 2 sockets
Aug 08 18:27:07 dist-test-slave-5k54 krb5kdc[15282](info): commencing operation
krb5kdc: starting...
W20260808 18:27:08.984361 23842 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Aug 08 18:27:09 dist-test-slave-5k54 krb5kdc[15282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213629, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:27:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:27:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.72.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
I20260808 18:27:09.114429 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:41127
--webserver_interface=127.23.72.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:34441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:41127
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:09.227376 15296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:09.227653 15296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:09.227694 15296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:09.231616 15296 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:27:09.231668 15296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:09.231688 15296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:09.231704 15296 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:27:09.231719 15296 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:27:09.236884 15296 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:34441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:41127
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:41127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.72.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15296
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:09.237836 15296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:09.238765 15296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:09.247151 15301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:09.247164 15302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:09.247459 15296 server_base.cc:1034] running on GCE node
W20260808 18:27:09.247607 15304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:09.248142 15296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:09.249198 15296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:09.250389 15296 hybrid_clock.cc:648] HybridClock initialized: now 1786213629250369 us; error 31 us; skew 500 ppm
Aug 08 18:27:09 dist-test-slave-5k54 krb5kdc[15282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213629, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:09.253279 15296 init.cc:377] Logged in from keytab as kudu/127.23.72.190@KRBTEST.COM (short username kudu)
I20260808 18:27:09.254118 15296 webserver.cc:460] Webserver started at http://127.23.72.190:36209/ using document root <none> and password file <none>
I20260808 18:27:09.254623 15296 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:09.254680 15296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:09.254839 15296 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:09.256867 15296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "b4bcca1677d44566874c9869526459a9"
format_stamp: "Formatted at 2026-08-08 18:27:09 on dist-test-slave-5k54"
server_key: "00d81b11e277eecc374bc5526b69d71b"
server_key_iv: "96905c0efc6aad02e0d2b3b44d407d04"
server_key_version: "encryptionkey@0"
I20260808 18:27:09.257391 15296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "b4bcca1677d44566874c9869526459a9"
format_stamp: "Formatted at 2026-08-08 18:27:09 on dist-test-slave-5k54"
server_key: "00d81b11e277eecc374bc5526b69d71b"
server_key_iv: "96905c0efc6aad02e0d2b3b44d407d04"
server_key_version: "encryptionkey@0"
I20260808 18:27:09.260980 15296 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 18:27:09.263347 15311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:09.264726 15296 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:27:09.264900 15296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "b4bcca1677d44566874c9869526459a9"
format_stamp: "Formatted at 2026-08-08 18:27:09 on dist-test-slave-5k54"
server_key: "00d81b11e277eecc374bc5526b69d71b"
server_key_iv: "96905c0efc6aad02e0d2b3b44d407d04"
server_key_version: "encryptionkey@0"
I20260808 18:27:09.264998 15296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:09.269971 15296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:09.273267 15296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:09.273417 15296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:09.280618 15296 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.190:41127
I20260808 18:27:09.280645 15371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.190:41127 every 8 connection(s)
I20260808 18:27:09.281549 15296 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:27:09.284298 15372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:09.289726 15372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4bcca1677d44566874c9869526459a9: Bootstrap starting.
I20260808 18:27:09.291217 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 15296
I20260808 18:27:09.291371 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:27:09.291685 23842 external_mini_cluster.cc:1428] Setting key 2af2313bc85dc4e61d61ef784143fd31
I20260808 18:27:09.292426 15372 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b4bcca1677d44566874c9869526459a9: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:09.293291 15372 log.cc:824] T 00000000000000000000000000000000 P b4bcca1677d44566874c9869526459a9: Log is configured to *not* fsync() on all Append() calls
I20260808 18:27:09.295507 15372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4bcca1677d44566874c9869526459a9: No bootstrap required, opened a new log
I20260808 18:27:09.296849 15375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:09.292797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34435 (local address 127.23.72.190:41127)
0808 18:27:09.293312 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:09.293333 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:09.293377 (+ 44us) server_negotiation.cc:407] Connection header received
0808 18:27:09.294173 (+ 796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:09.294196 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:09.294537 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:09.294898 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:09.295698 (+ 800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:09.295703 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:09.295714 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:27:09.295725 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:09.295781 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:27:09.295800 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:09.295926 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:09.296028 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 18:27:09.296180 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":159,"threads_started":1}
I20260808 18:27:09.299612 23842 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:27:09.299742 15372 raft_consensus.cc:348] T 00000000000000000000000000000000 P b4bcca1677d44566874c9869526459a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4bcca1677d44566874c9869526459a9" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 41127 } }
I20260808 18:27:09.300006 15372 raft_consensus.cc:374] T 00000000000000000000000000000000 P b4bcca1677d44566874c9869526459a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:09.300096 15372 raft_consensus.cc:729] T 00000000000000000000000000000000 P b4bcca1677d44566874c9869526459a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4bcca1677d44566874c9869526459a9, State: Initialized, Role: FOLLOWER
I20260808 18:27:09.300508 15372 consensus_queue.cc:260] T 00000000000000000000000000000000 P b4bcca1677d44566874c9869526459a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4bcca1677d44566874c9869526459a9" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 41127 } }
I20260808 18:27:09.300655 15372 raft_consensus.cc:388] T 00000000000000000000000000000000 P b4bcca1677d44566874c9869526459a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:27:09.300747 15372 raft_consensus.cc:482] T 00000000000000000000000000000000 P b4bcca1677d44566874c9869526459a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:27:09.300866 15372 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b4bcca1677d44566874c9869526459a9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:09.301959 15372 raft_consensus.cc:504] T 00000000000000000000000000000000 P b4bcca1677d44566874c9869526459a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4bcca1677d44566874c9869526459a9" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 41127 } }
I20260808 18:27:09.302341 15372 leader_election.cc:302] T 00000000000000000000000000000000 P b4bcca1677d44566874c9869526459a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b4bcca1677d44566874c9869526459a9; no voters:
I20260808 18:27:09.302670 15372 leader_election.cc:288] T 00000000000000000000000000000000 P b4bcca1677d44566874c9869526459a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:27:09.302789 15382 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b4bcca1677d44566874c9869526459a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:27:09.303002 15382 raft_consensus.cc:686] T 00000000000000000000000000000000 P b4bcca1677d44566874c9869526459a9 [term 1 LEADER]: Becoming Leader. State: Replica: b4bcca1677d44566874c9869526459a9, State: Running, Role: LEADER
I20260808 18:27:09.303372 15382 consensus_queue.cc:237] T 00000000000000000000000000000000 P b4bcca1677d44566874c9869526459a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4bcca1677d44566874c9869526459a9" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 41127 } }
I20260808 18:27:09.303822 15375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:09.301857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52695 (local address 127.23.72.190:41127)
0808 18:27:09.301987 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:09.301991 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:09.302208 (+ 217us) server_negotiation.cc:407] Connection header received
0808 18:27:09.302303 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:09.302306 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:09.302338 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:09.302419 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:09.303247 (+ 828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:09.303251 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:09.303253 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:27:09.303264 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:09.303290 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:27:09.303301 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:09.303383 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:09.303678 (+ 295us) server_negotiation.cc:299] Negotiation successful
0808 18:27:09.303716 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 18:27:09.304064 15372 sys_catalog.cc:564] T 00000000000000000000000000000000 P b4bcca1677d44566874c9869526459a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:27:09.305244 15383 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4bcca1677d44566874c9869526459a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b4bcca1677d44566874c9869526459a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4bcca1677d44566874c9869526459a9" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 41127 } } }
I20260808 18:27:09.305372 15383 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4bcca1677d44566874c9869526459a9 [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:09.305584 15384 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4bcca1677d44566874c9869526459a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4bcca1677d44566874c9869526459a9. Latest consensus state: current_term: 1 leader_uuid: "b4bcca1677d44566874c9869526459a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4bcca1677d44566874c9869526459a9" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 41127 } } }
I20260808 18:27:09.305792 15384 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4bcca1677d44566874c9869526459a9 [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:09.306078 15392 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:27:09.308890 15392 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:27:09.314092 15392 catalog_manager.cc:1347] Generated new cluster ID: 69fdf872709e42e5b1589960ed820e14
I20260808 18:27:09.314155 15392 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:27:09.344017 15392 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:27:09.344362 15392 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:27:09.365706 15392 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b4bcca1677d44566874c9869526459a9: Generated new TSK 0
I20260808 18:27:09.366457 15392 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:27:09.373507 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 15296
2026-08-08T18:27:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1461 ms)
[----------] 18 tests from AuthTokenIssuingTest (26339 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:27:09 dist-test-slave-5k54 krb5kdc[15404](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:27:09 dist-test-slave-5k54 krb5kdc[15404](info): set up 2 sockets
Aug 08 18:27:09 dist-test-slave-5k54 krb5kdc[15404](info): commencing operation
krb5kdc: starting...
W20260808 18:27:10.456461 23842 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Aug 08 18:27:10 dist-test-slave-5k54 krb5kdc[15404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213630, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:27:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:27:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.72.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
I20260808 18:27:10.586614 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:43387
--webserver_interface=127.23.72.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:40255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:43387,127.23.72.189:46719,127.23.72.188:44469
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:10.700692 15418 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 18:27:10.700964 15418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:10.701012 15418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:10.701105 15418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:10.705235 15418 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:27:10.705293 15418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:10.705312 15418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:10.705350 15418 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:27:10.705380 15418 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:27:10.710610 15418 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:40255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:43387,127.23.72.189:46719,127.23.72.188:44469
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:43387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.72.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15418
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:10.711688 15418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:10.712769 15418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:10.721191 15423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:10.721210 15418 server_base.cc:1034] running on GCE node
W20260808 18:27:10.721257 15427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:10.721190 15424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:10.722103 15418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:10.723111 15418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:10.724272 15418 hybrid_clock.cc:648] HybridClock initialized: now 1786213630724254 us; error 29 us; skew 500 ppm
Aug 08 18:27:10 dist-test-slave-5k54 krb5kdc[15404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213630, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:10.726985 15418 init.cc:377] Logged in from keytab as kudu/127.23.72.190@KRBTEST.COM (short username kudu)
I20260808 18:27:10.727903 15418 webserver.cc:460] Webserver started at http://127.23.72.190:37585/ using document root <none> and password file <none>
I20260808 18:27:10.728407 15418 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:10.728463 15418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:10.728621 15418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:10.730571 15418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "d0a407ba2ea5402da1f6d793183db2ec"
format_stamp: "Formatted at 2026-08-08 18:27:10 on dist-test-slave-5k54"
server_key: "b4986d4a49cb3c09c0fe259461674035"
server_key_iv: "a639c91ee19f7355d7284ad81c526ab9"
server_key_version: "encryptionkey@0"
I20260808 18:27:10.731073 15418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "d0a407ba2ea5402da1f6d793183db2ec"
format_stamp: "Formatted at 2026-08-08 18:27:10 on dist-test-slave-5k54"
server_key: "b4986d4a49cb3c09c0fe259461674035"
server_key_iv: "a639c91ee19f7355d7284ad81c526ab9"
server_key_version: "encryptionkey@0"
I20260808 18:27:10.734673 15418 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 18:27:10.737005 15434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:10.738324 15418 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:27:10.738481 15418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "d0a407ba2ea5402da1f6d793183db2ec"
format_stamp: "Formatted at 2026-08-08 18:27:10 on dist-test-slave-5k54"
server_key: "b4986d4a49cb3c09c0fe259461674035"
server_key_iv: "a639c91ee19f7355d7284ad81c526ab9"
server_key_version: "encryptionkey@0"
I20260808 18:27:10.738595 15418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:10.751070 15418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:10.754110 15418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:10.754314 15418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:10.761686 15418 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.190:43387
I20260808 18:27:10.761971 15490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.190:43387 every 8 connection(s)
I20260808 18:27:10.762655 15418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:27:10.763552 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 15418
I20260808 18:27:10.763696 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:27:10.764026 23842 external_mini_cluster.cc:1428] Setting key 9eb2476063e11623ead40fbe4b4d6a1f
I20260808 18:27:10.766136 15491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:10.770219 15491 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.72.190" port: 43387 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.23.72.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.189@KRBTEST.COM" created.
Aug 08 18:27:10 dist-test-slave-5k54 krb5kdc[15404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213630, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:27:10.786806 15497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:10.774101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51850 (local address 127.23.72.190:43387)
0808 18:27:10.774445 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:10.774456 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:10.774500 (+ 44us) server_negotiation.cc:407] Connection header received
0808 18:27:10.774920 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:10.774925 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:10.775108 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:10.775309 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:10.776316 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:10.777517 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:10.778583 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:10.779090 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:10.779209 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:10.781693 (+ 2484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:10.781699 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:10.781708 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:10.781728 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:10.783838 (+ 2110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:10.784323 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:10.784332 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:10.784335 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:10.784482 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:10.784804 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:10.784809 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:10.784811 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:10.785338 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:10.785484 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:10.785861 (+ 377us) server_negotiation.cc:299] Negotiation successful
0808 18:27:10.786154 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":67,"server-negotiator.queue_time_us":226,"thread_start_us":116,"threads_started":1}
I20260808 18:27:10.786813 15496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:10.772934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:43387 (local address 127.0.0.1:51850)
0808 18:27:10.773560 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:10.773604 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:10.774580 (+ 976us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:10.775309 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:10.775314 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:10.775787 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:10.776168 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:10.776183 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:10.777671 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:10.777674 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:10.778453 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:10.778463 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:10.779271 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:10.779276 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:10.779442 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:10.780047 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:10.780073 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:10.781541 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:10.783960 (+ 2419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:10.783968 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:10.783976 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:10.784210 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:10.784588 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:10.784590 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:10.784592 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:10.784702 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:10.785469 (+ 767us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:10.785474 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:10.785598 (+ 124us) client_negotiation.cc:769] Sending connection context
0808 18:27:10.785733 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 18:27:10.786018 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":217,"threads_started":1}
I20260808 18:27:10.789361 15491 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46719 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:27:10.790167 15496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:10.789864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.189:46719 (local address 127.0.0.1:50592)
0808 18:27:10.790026 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:10.790106 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.189:46719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260808 18:27:10.790462 15439 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.72.189:46719: connect: Connection refused (error 111)
I20260808 18:27:10.791381 15496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:10.791168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.189:46719 (local address 127.0.0.1:50608)
0808 18:27:10.791281 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:10.791329 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.189:46719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 18:27:10.791625 15491 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.72.189:46719: Network error: Client connection negotiation failed: client connection to 127.23.72.189:46719: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.189.keytab.
Entry for principal kudu/127.23.72.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.189.keytab.
I20260808 18:27:10.795063 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.189:46719
--webserver_interface=127.23.72.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.189.keytab
--principal=kudu/127.23.72.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:40255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:43387,127.23.72.189:46719,127.23.72.188:44469
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:27:10.840936 15491 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46719 } attempt: 1
I20260808 18:27:10.841929 15496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:10.841622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.189:46719 (local address 127.0.0.1:50624)
0808 18:27:10.841778 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:10.841845 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.189:46719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 18:27:10.842727 15496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:10.842523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.189:46719 (local address 127.0.0.1:50636)
0808 18:27:10.842625 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:10.842671 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.189:46719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 18:27:10.842860 15491 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.72.189:46719: Network error: Client connection negotiation failed: client connection to 127.23.72.189:46719: connect: Connection refused (error 111)
I20260808 18:27:10.911055 15491 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46719 } attempt: 2
W20260808 18:27:10.911038 15500 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 18:27:10.911473 15500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:10.911536 15500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:10.911592 15500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 18:27:10.912024 15496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:10.911709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.189:46719 (local address 127.0.0.1:50644)
0808 18:27:10.911876 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:10.911945 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.189:46719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 18:27:10.912799 15496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:10.912609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.189:46719 (local address 127.0.0.1:50656)
0808 18:27:10.912707 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:10.912749 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.189:46719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 18:27:10.912935 15491 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.72.189:46719: Network error: Client connection negotiation failed: client connection to 127.23.72.189:46719: connect: Connection refused (error 111)
W20260808 18:27:10.916116 15500 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:27:10.916167 15500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:10.916188 15500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:10.916241 15500 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:27:10.916276 15500 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:27:10.921567 15500 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:40255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:43387,127.23.72.189:46719,127.23.72.188:44469
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.189.keytab
--principal=kudu/127.23.72.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.189:46719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.23.72.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:10.922641 15500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:10.923678 15500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:10.931753 15506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:10.931756 15505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:10.931753 15508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:10.932044 15500 server_base.cc:1034] running on GCE node
I20260808 18:27:10.932682 15500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:10.933753 15500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:10.934922 15500 hybrid_clock.cc:648] HybridClock initialized: now 1786213630934912 us; error 47 us; skew 500 ppm
Aug 08 18:27:10 dist-test-slave-5k54 krb5kdc[15404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213630, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:10.937706 15500 init.cc:377] Logged in from keytab as kudu/127.23.72.189@KRBTEST.COM (short username kudu)
I20260808 18:27:10.938621 15500 webserver.cc:460] Webserver started at http://127.23.72.189:45995/ using document root <none> and password file <none>
I20260808 18:27:10.939177 15500 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:10.939267 15500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:10.939577 15500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:10.941617 15500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-1/data/instance:
uuid: "3ec28634d339405bad3ed43be0681772"
format_stamp: "Formatted at 2026-08-08 18:27:10 on dist-test-slave-5k54"
server_key: "72d5732842ecc91fd639e4e3828d0ef1"
server_key_iv: "a68cc705a773b4a967ab96b3dac13835"
server_key_version: "encryptionkey@0"
I20260808 18:27:10.942320 15500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-1/wal/instance:
uuid: "3ec28634d339405bad3ed43be0681772"
format_stamp: "Formatted at 2026-08-08 18:27:10 on dist-test-slave-5k54"
server_key: "72d5732842ecc91fd639e4e3828d0ef1"
server_key_iv: "a68cc705a773b4a967ab96b3dac13835"
server_key_version: "encryptionkey@0"
I20260808 18:27:10.945974 15500 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260808 18:27:10.948401 15516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:10.949618 15500 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:27:10.949754 15500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-1/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-1/wal
uuid: "3ec28634d339405bad3ed43be0681772"
format_stamp: "Formatted at 2026-08-08 18:27:10 on dist-test-slave-5k54"
server_key: "72d5732842ecc91fd639e4e3828d0ef1"
server_key_iv: "a68cc705a773b4a967ab96b3dac13835"
server_key_version: "encryptionkey@0"
I20260808 18:27:10.949862 15500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:10.975670 15500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:10.978646 15500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:10.978873 15500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:10.985925 15500 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.189:46719
I20260808 18:27:10.986219 15570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.189:46719 every 8 connection(s)
I20260808 18:27:10.986877 15500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-1/data/info.pb
I20260808 18:27:10.989638 15572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:10.993085 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 15500
I20260808 18:27:10.993237 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-1/wal/instance
I20260808 18:27:10.993521 23842 external_mini_cluster.cc:1428] Setting key 58ff590268c6e335fc13cec9a8a724db
I20260808 18:27:10.993362 15572 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.72.190" port: 43387 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.23.72.188@KRBTEST.COM; defaulting to no policy
I20260808 18:27:11.004081 15491 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46719 } attempt: 3
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.188@KRBTEST.COM" created.
Aug 08 18:27:11 dist-test-slave-5k54 krb5kdc[15404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213630, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.189@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
Aug 08 18:27:11 dist-test-slave-5k54 krb5kdc[15404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213630, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for kudu/127.23.72.189@KRBTEST.COM
I20260808 18:27:11.013145 15497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:10.995237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51862 (local address 127.23.72.190:43387)
0808 18:27:10.995376 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:10.995381 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:10.996441 (+ 1060us) server_negotiation.cc:407] Connection header received
0808 18:27:10.997721 (+ 1280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:10.997726 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:10.997871 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:10.997992 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:10.999682 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:11.000328 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:11.002134 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:11.002403 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:11.002513 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:11.007731 (+ 5218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:11.007734 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:11.007736 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:11.007760 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:11.010386 (+ 2626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:11.011032 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:11.011039 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:11.011045 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:11.011152 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:11.011623 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:11.011627 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:11.011628 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:11.011900 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:11.012020 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:11.012802 (+ 782us) server_negotiation.cc:299] Negotiation successful
0808 18:27:11.012937 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 18:27:11.014531 15575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:10.995766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:43387 (local address 127.0.0.1:51862)
0808 18:27:10.996276 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:10.996318 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:10.997404 (+ 1086us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:10.998176 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:10.998187 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:10.998752 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:10.999508 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:10.999532 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:11.000473 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:11.000480 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:11.001985 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:11.001998 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:11.002605 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:11.002610 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:11.002828 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:11.003544 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:11.003570 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:11.007574 (+ 4004us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:11.010509 (+ 2935us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:11.010515 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:11.010531 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:11.010889 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:11.011253 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:11.011259 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:11.011263 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:11.011527 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:11.012051 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:11.012063 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:11.012449 (+ 386us) client_negotiation.cc:769] Sending connection context
0808 18:27:11.012706 (+ 257us) client_negotiation.cc:240] Negotiation successful
0808 18:27:11.013169 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":145,"threads_started":1}
I20260808 18:27:11.015079 15496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:11.004543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.189:46719 (local address 127.0.0.1:50662)
0808 18:27:11.004676 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:11.004691 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:11.004780 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:11.005376 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:11.005380 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:11.005601 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:11.005748 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:11.005756 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:11.006959 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:11.006963 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:11.008007 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:11.008024 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:11.008731 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:11.008735 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:11.008859 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:11.009340 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:11.009357 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:11.011047 (+ 1690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:11.013516 (+ 2469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:11.013520 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:11.013522 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:11.013751 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:11.014033 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:11.014036 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:11.014038 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:11.014125 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:11.014574 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:11.014577 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:11.014677 (+ 100us) client_negotiation.cc:769] Sending connection context
0808 18:27:11.014771 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 18:27:11.014932 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 18:27:11.015666 15576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:11.004654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50662 (local address 127.23.72.189:46719)
0808 18:27:11.004962 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:11.004981 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:11.005002 (+ 21us) server_negotiation.cc:407] Connection header received
0808 18:27:11.005036 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:11.005039 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:11.005195 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:11.005329 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:11.005886 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:11.006808 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:11.008152 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:11.008599 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:11.008732 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:11.011174 (+ 2442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:11.011184 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:11.011198 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:11.011223 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:11.013383 (+ 2160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:11.013862 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:11.013866 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:11.013868 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:11.013937 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:11.014217 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:11.014221 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:11.014223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:11.014464 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:11.014604 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:11.015316 (+ 712us) server_negotiation.cc:299] Negotiation successful
0808 18:27:11.015483 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":127,"threads_started":1}
I20260808 18:27:11.016355 15572 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46719 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:27:11.017618 15491 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.72.188" port: 44469 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:27:11.018395 15496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:11.018120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:44469 (local address 127.0.0.1:44514)
0808 18:27:11.018291 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:11.018341 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.188:44469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260808 18:27:11.019351 15496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:11.018975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:44469 (local address 127.0.0.1:44530)
0808 18:27:11.019246 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:11.019303 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.188:44469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207}
W20260808 18:27:11.019515 15491 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.72.188:44469: Network error: Client connection negotiation failed: client connection to 127.23.72.188:44469: connect: Connection refused (error 111)
Aug 08 18:27:11 dist-test-slave-5k54 krb5kdc[15404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213630, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.189@KRBTEST.COM for kudu/127.23.72.189@KRBTEST.COM
I20260808 18:27:11.025010 15575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:11.016827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.189:46719 (local address 127.0.0.1:50678)
0808 18:27:11.016915 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:11.016928 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:11.016999 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:11.017238 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:11.017241 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:11.017446 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:11.017551 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:11.017557 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:11.018510 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:11.018513 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:11.019209 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:11.019219 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:11.019792 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:11.019796 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:11.019886 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:11.020345 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:11.020365 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:11.021581 (+ 1216us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:11.023504 (+ 1923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:11.023507 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:11.023509 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:11.023763 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:11.024076 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:11.024078 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:11.024079 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:11.024152 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:11.024526 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:11.024528 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:11.024604 (+ 76us) client_negotiation.cc:769] Sending connection context
0808 18:27:11.024686 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 18:27:11.024862 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260808 18:27:11.025039 15576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:11.016963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50678 (local address 127.23.72.189:46719)
0808 18:27:11.017067 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:11.017072 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:11.017083 (+ 11us) server_negotiation.cc:407] Connection header received
0808 18:27:11.017107 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:11.017110 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:11.017149 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:11.017229 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:11.017642 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:11.018367 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:11.019322 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:11.019698 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:11.019795 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:11.021714 (+ 1919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:11.021717 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:11.021719 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:11.021743 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:11.023353 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:11.023888 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:11.023894 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:11.023896 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:11.023986 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:11.024233 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:11.024235 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:11.024236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:11.024437 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:11.024517 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:11.024788 (+ 271us) server_negotiation.cc:299] Negotiation successful
0808 18:27:11.024923 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":37}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.188.keytab.
Entry for principal kudu/127.23.72.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.188.keytab.
I20260808 18:27:11.025774 15572 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.72.188" port: 44469 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:27:11.026525 15575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:11.026224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:44469 (local address 127.0.0.1:44544)
0808 18:27:11.026363 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:11.026456 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.188:44469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 18:27:11.026548 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.188:44469
--webserver_interface=127.23.72.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.188.keytab
--principal=kudu/127.23.72.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:40255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:43387,127.23.72.189:46719,127.23.72.188:44469
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:11.026777 15517 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.72.188:44469: connect: Connection refused (error 111)
I20260808 18:27:11.027782 15575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:11.027494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:44469 (local address 127.0.0.1:44550)
0808 18:27:11.027645 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:11.027706 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.188:44469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260808 18:27:11.028016 15572 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.72.188:44469: Network error: Client connection negotiation failed: client connection to 127.23.72.188:44469: connect: Connection refused (error 111)
I20260808 18:27:11.050685 15491 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.72.188" port: 44469 } attempt: 1
I20260808 18:27:11.051642 15496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:11.051292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:44469 (local address 127.0.0.1:44558)
0808 18:27:11.051496 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:11.051561 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.188:44469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260808 18:27:11.052448 15496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:11.052241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:44469 (local address 127.0.0.1:44562)
0808 18:27:11.052341 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:11.052385 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.188:44469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 18:27:11.052595 15491 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.72.188:44469: Network error: Client connection negotiation failed: client connection to 127.23.72.188:44469: connect: Connection refused (error 111)
I20260808 18:27:11.077327 15572 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.72.188" port: 44469 } attempt: 1
I20260808 18:27:11.078311 15575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:11.077943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:44469 (local address 127.0.0.1:44576)
0808 18:27:11.078136 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:11.078211 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.188:44469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260808 18:27:11.079167 15575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:11.078955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:44469 (local address 127.0.0.1:44582)
0808 18:27:11.079060 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:11.079115 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.188:44469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 18:27:11.079308 15572 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.72.188:44469: Network error: Client connection negotiation failed: client connection to 127.23.72.188:44469: connect: Connection refused (error 111)
I20260808 18:27:11.127799 15491 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.72.188" port: 44469 } attempt: 2
I20260808 18:27:11.128778 15496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:11.128425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:44469 (local address 127.0.0.1:44598)
0808 18:27:11.128603 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:11.128681 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.188:44469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260808 18:27:11.129551 15496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:11.129336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:44469 (local address 127.0.0.1:44604)
0808 18:27:11.129444 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:11.129496 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.188:44469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 18:27:11.129684 15491 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.72.188:44469: Network error: Client connection negotiation failed: client connection to 127.23.72.188:44469: connect: Connection refused (error 111)
W20260808 18:27:11.140756 15579 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 18:27:11.141086 15579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:11.141147 15579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:11.141224 15579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:11.145509 15579 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:27:11.145591 15579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:11.145630 15579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:11.145669 15579 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:27:11.145701 15579 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:27:11.147514 15572 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.72.188" port: 44469 } attempt: 2
I20260808 18:27:11.148332 15575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:11.148026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:44469 (local address 127.0.0.1:44612)
0808 18:27:11.148179 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:11.148248 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.188:44469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260808 18:27:11.149109 15575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:11.148896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:44469 (local address 127.0.0.1:44628)
0808 18:27:11.149009 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:11.149059 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.188:44469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 18:27:11.149246 15572 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.72.188:44469: Network error: Client connection negotiation failed: client connection to 127.23.72.188:44469: connect: Connection refused (error 111)
I20260808 18:27:11.151233 15579 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:40255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:43387,127.23.72.189:46719,127.23.72.188:44469
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.188.keytab
--principal=kudu/127.23.72.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.188:44469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.23.72.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15579
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:11.152318 15579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:11.153362 15579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 18:27:11.161966 15579 server_base.cc:1034] running on GCE node
W20260808 18:27:11.161913 15584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:11.161913 15587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:11.161933 15585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:11.162767 15579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:11.163954 15579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:11.165150 15579 hybrid_clock.cc:648] HybridClock initialized: now 1786213631165136 us; error 26 us; skew 500 ppm
Aug 08 18:27:11 dist-test-slave-5k54 krb5kdc[15404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213631, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:11.167951 15579 init.cc:377] Logged in from keytab as kudu/127.23.72.188@KRBTEST.COM (short username kudu)
I20260808 18:27:11.168751 15579 webserver.cc:460] Webserver started at http://127.23.72.188:34563/ using document root <none> and password file <none>
I20260808 18:27:11.169262 15579 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:11.169322 15579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:11.169483 15579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:11.171533 15579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-2/data/instance:
uuid: "dd0c81b88967477a8eb7f6bfd8f529f4"
format_stamp: "Formatted at 2026-08-08 18:27:11 on dist-test-slave-5k54"
server_key: "f2983cdfdd21810d81277d81b22043fe"
server_key_iv: "0f98d7ba327032ed0fae669daae6a2da"
server_key_version: "encryptionkey@0"
I20260808 18:27:11.172060 15579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-2/wal/instance:
uuid: "dd0c81b88967477a8eb7f6bfd8f529f4"
format_stamp: "Formatted at 2026-08-08 18:27:11 on dist-test-slave-5k54"
server_key: "f2983cdfdd21810d81277d81b22043fe"
server_key_iv: "0f98d7ba327032ed0fae669daae6a2da"
server_key_version: "encryptionkey@0"
I20260808 18:27:11.175745 15579 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20260808 18:27:11.178174 15595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:11.179586 15579 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:27:11.179716 15579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-2/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-2/wal
uuid: "dd0c81b88967477a8eb7f6bfd8f529f4"
format_stamp: "Formatted at 2026-08-08 18:27:11 on dist-test-slave-5k54"
server_key: "f2983cdfdd21810d81277d81b22043fe"
server_key_iv: "0f98d7ba327032ed0fae669daae6a2da"
server_key_version: "encryptionkey@0"
I20260808 18:27:11.179836 15579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:11.199024 15579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:11.201975 15579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:11.202180 15579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:11.209625 15579 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.188:44469
I20260808 18:27:11.209897 15652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.188:44469 every 8 connection(s)
I20260808 18:27:11.210619 15579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-2/data/info.pb
I20260808 18:27:11.213400 15654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:11.213832 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 15579
I20260808 18:27:11.213976 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213345431840-23842-0/minicluster-data/master-2/wal/instance
I20260808 18:27:11.214288 23842 external_mini_cluster.cc:1428] Setting key d8b216f5f70bab27ab0d57ab980a69d4
I20260808 18:27:11.218282 15654 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.72.190" port: 43387 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 18:27:11 dist-test-slave-5k54 krb5kdc[15404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213630, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.72.188@KRBTEST.COM
Aug 08 18:27:11 dist-test-slave-5k54 krb5kdc[15404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213631, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.188@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:27:11.228852 15491 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.72.188" port: 44469 } attempt: 3
I20260808 18:27:11.229593 15657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:11.215709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44640 (local address 127.23.72.188:44469)
0808 18:27:11.216102 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:11.216116 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:11.216152 (+ 36us) server_negotiation.cc:407] Connection header received
0808 18:27:11.216818 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:11.216835 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:11.217158 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:11.217440 (+ 282us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:11.218379 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:11.219471 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:11.220476 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:11.221156 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:11.221316 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:11.224430 (+ 3114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:11.224435 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:11.224444 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:11.224470 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:11.226312 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:11.226772 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:11.226778 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:11.226783 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:11.226904 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:11.227251 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:11.227254 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:11.227255 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:11.227785 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:11.227977 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:11.228302 (+ 325us) server_negotiation.cc:299] Negotiation successful
0808 18:27:11.228600 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":125,"threads_started":1}
I20260808 18:27:11.230691 15658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:11.220187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:43387 (local address 127.0.0.1:51870)
0808 18:27:11.220571 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:11.220616 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:11.220729 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:11.221023 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:11.221028 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:11.221513 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:11.221699 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:11.221708 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:11.222880 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:11.222883 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:11.223903 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:11.223918 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:11.224551 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:11.224557 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:11.224680 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:11.225171 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:11.225190 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:11.226897 (+ 1707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:11.228903 (+ 2006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:11.228911 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:11.228917 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:11.229190 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:11.229527 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:11.229530 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:11.229532 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:11.229621 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:11.230081 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:11.230085 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:11.230215 (+ 130us) client_negotiation.cc:769] Sending connection context
0808 18:27:11.230353 (+ 138us) client_negotiation.cc:240] Negotiation successful
0808 18:27:11.230493 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":109,"threads_started":1}
I20260808 18:27:11.231015 15497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:11.220252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51870 (local address 127.23.72.190:43387)
0808 18:27:11.220421 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:11.220426 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:11.220698 (+ 272us) server_negotiation.cc:407] Connection header received
0808 18:27:11.220826 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:11.220831 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:11.220886 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:11.220953 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:11.221905 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:11.222743 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:11.224033 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:11.224441 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:11.224523 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:11.227065 (+ 2542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:11.227069 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:11.227074 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:11.227107 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:11.228773 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:11.229312 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:11.229316 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:11.229318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:11.229402 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:11.229725 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:11.229728 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:11.229730 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:11.229943 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:11.230016 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:11.230448 (+ 432us) server_negotiation.cc:299] Negotiation successful
0808 18:27:11.230571 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260808 18:27:11.232136 15654 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46719 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 18:27:11 dist-test-slave-5k54 krb5kdc[15404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213630, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for kudu/127.23.72.188@KRBTEST.COM
Aug 08 18:27:11 dist-test-slave-5k54 krb5kdc[15404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213630, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
Aug 08 18:27:11 dist-test-slave-5k54 krb5kdc[15404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213631, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.188@KRBTEST.COM for kudu/127.23.72.189@KRBTEST.COM
I20260808 18:27:11.240339 15659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:11.229312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44646 (local address 127.23.72.188:44469)
0808 18:27:11.229701 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:11.229714 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:11.229728 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:27:11.230555 (+ 827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:11.230558 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:11.230602 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:11.231157 (+ 555us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:11.231238 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:11.232011 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:11.233046 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:11.233293 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:11.233397 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:11.235539 (+ 2142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:11.235544 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:11.235548 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:11.235574 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:11.238014 (+ 2440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:11.238511 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:11.238517 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:11.238519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:11.238606 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:11.238947 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:11.238952 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:11.238957 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:11.239520 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:11.239665 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:11.239972 (+ 307us) server_negotiation.cc:299] Negotiation successful
0808 18:27:11.240129 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":152,"threads_started":1}
I20260808 18:27:11.240432 15572 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.72.188" port: 44469 } attempt: 3
I20260808 18:27:11.240528 15496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:11.229263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:44469 (local address 127.0.0.1:44646)
0808 18:27:11.229494 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:11.229510 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:11.229589 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:11.230737 (+ 1148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:11.230740 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:11.230932 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:11.231027 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:11.231037 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:11.232136 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:11.232140 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:11.232887 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:11.232896 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:11.233399 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:11.233403 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:11.233489 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:11.233955 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:11.233966 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:11.235393 (+ 1427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:11.238134 (+ 2741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:11.238141 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:11.238144 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:11.238395 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:11.238715 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:11.238721 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:11.238724 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:11.238816 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:11.239658 (+ 842us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:11.239662 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:11.239769 (+ 107us) client_negotiation.cc:769] Sending connection context
0808 18:27:11.239889 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 18:27:11.240139 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128}
I20260808 18:27:11.242269 15497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:11.232345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51882 (local address 127.23.72.190:43387)
0808 18:27:11.232478 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:11.232481 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:11.232492 (+ 11us) server_negotiation.cc:407] Connection header received
0808 18:27:11.232517 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:11.232519 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:11.232547 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:11.232634 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:11.233308 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:11.234014 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:11.234778 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:11.235002 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:11.235221 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:11.238510 (+ 3289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:11.238513 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:11.238515 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:11.238537 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:11.240255 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:11.240763 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:11.240767 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:11.240768 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:11.240852 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:11.241189 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:11.241192 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:11.241193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:11.241428 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:11.241661 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:11.242032 (+ 371us) server_negotiation.cc:299] Negotiation successful
0808 18:27:11.242142 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 18:27:11.244102 15491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0a407ba2ea5402da1f6d793183db2ec: Bootstrap starting.
I20260808 18:27:11.244323 15576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:11.232880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50692 (local address 127.23.72.189:46719)
0808 18:27:11.233043 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:11.233048 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:11.233455 (+ 407us) server_negotiation.cc:407] Connection header received
0808 18:27:11.233638 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:11.233644 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:11.233700 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:11.233809 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:11.235270 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:11.235886 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:11.236647 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:11.236897 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:11.237010 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:11.239194 (+ 2184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:11.239198 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:11.239201 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:11.239229 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:11.242046 (+ 2817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:11.243005 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:11.243008 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:11.243010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:11.243093 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:11.243382 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:11.243385 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:11.243387 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:11.243633 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:11.243703 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:11.244037 (+ 334us) server_negotiation.cc:299] Negotiation successful
0808 18:27:11.244145 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 18:27:11.244834 15658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:11.232612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.189:46719 (local address 127.0.0.1:50692)
0808 18:27:11.233381 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:11.233394 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:11.233520 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:11.233848 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:11.233852 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:11.234530 (+ 678us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:11.235121 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:11.235133 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:11.236016 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:11.236018 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:11.236517 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:11.236523 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:11.237015 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:11.237018 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:11.237118 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:11.237508 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:11.237525 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:11.239064 (+ 1539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:11.242712 (+ 3648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:11.242714 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:11.242715 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:11.242894 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:11.243182 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:11.243185 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:11.243187 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:11.243279 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:11.243777 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:11.243780 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:11.243859 (+ 79us) client_negotiation.cc:769] Sending connection context
0808 18:27:11.244570 (+ 711us) client_negotiation.cc:240] Negotiation successful
0808 18:27:11.244704 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":703}
I20260808 18:27:11.245524 15654 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.72.188" port: 44469 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 18:27:11 dist-test-slave-5k54 krb5kdc[15404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213630, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.189@KRBTEST.COM for kudu/127.23.72.188@KRBTEST.COM
I20260808 18:27:11.248188 15491 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0a407ba2ea5402da1f6d793183db2ec: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:11.249174 15491 log.cc:824] T 00000000000000000000000000000000 P d0a407ba2ea5402da1f6d793183db2ec: Log is configured to *not* fsync() on all Append() calls
I20260808 18:27:11.250424 15575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:11.240822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:44469 (local address 127.0.0.1:44652)
0808 18:27:11.241045 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:11.241060 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:11.241137 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:11.241491 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:11.241493 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:11.242276 (+ 783us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:11.242379 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:11.242386 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:11.243303 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:11.243309 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:11.244138 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:11.244146 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:11.244700 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:11.244704 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:11.244810 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:11.245269 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:11.245288 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:11.246850 (+ 1562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:11.248813 (+ 1963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:11.248816 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:11.248818 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:11.249082 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:11.249391 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:11.249394 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:11.249396 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:11.249504 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:11.249942 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:11.249945 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:11.250047 (+ 102us) client_negotiation.cc:769] Sending connection context
0808 18:27:11.250135 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 18:27:11.250268 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147}
I20260808 18:27:11.251220 15659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:11.240916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44652 (local address 127.23.72.188:44469)
0808 18:27:11.241279 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:11.241283 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:11.241295 (+ 12us) server_negotiation.cc:407] Connection header received
0808 18:27:11.241325 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:11.241328 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:11.241368 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:11.241538 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:11.242504 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:11.243208 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:11.244256 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:11.244563 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:11.245027 (+ 464us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:11.246995 (+ 1968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:11.246998 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:11.247000 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:11.247023 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:11.248674 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:11.249220 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:11.249223 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:11.249225 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:11.249307 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:11.249609 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:11.249612 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:11.249614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:11.249817 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:11.250903 (+ 1086us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:11.250981 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 18:27:11.251090 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289}
I20260808 18:27:11.251652 15491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0a407ba2ea5402da1f6d793183db2ec: No bootstrap required, opened a new log
Aug 08 18:27:11 dist-test-slave-5k54 krb5kdc[15404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213630, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.72.189@KRBTEST.COM
I20260808 18:27:11.254920 15572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ec28634d339405bad3ed43be0681772: Bootstrap starting.
Aug 08 18:27:11 dist-test-slave-5k54 krb5kdc[15404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213631, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.188@KRBTEST.COM for kudu/127.23.72.188@KRBTEST.COM
I20260808 18:27:11.256033 15491 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0a407ba2ea5402da1f6d793183db2ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0a407ba2ea5402da1f6d793183db2ec" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 43387 } } peers { permanent_uuid: "3ec28634d339405bad3ed43be0681772" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46719 } } peers { permanent_uuid: "dd0c81b88967477a8eb7f6bfd8f529f4" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 44469 } }
I20260808 18:27:11.256266 15491 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0a407ba2ea5402da1f6d793183db2ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0a407ba2ea5402da1f6d793183db2ec, State: Initialized, Role: FOLLOWER
I20260808 18:27:11.256596 15491 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0a407ba2ea5402da1f6d793183db2ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0a407ba2ea5402da1f6d793183db2ec" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 43387 } } peers { permanent_uuid: "3ec28634d339405bad3ed43be0681772" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46719 } } peers { permanent_uuid: "dd0c81b88967477a8eb7f6bfd8f529f4" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 44469 } }
I20260808 18:27:11.256810 15576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:11.246468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50708 (local address 127.23.72.189:46719)
0808 18:27:11.246612 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:11.246617 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:11.246659 (+ 42us) server_negotiation.cc:407] Connection header received
0808 18:27:11.246788 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:11.246795 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:11.246833 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:11.246926 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:11.247670 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:11.248338 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:11.249103 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:11.249411 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:11.249513 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:11.252418 (+ 2905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:11.252422 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:11.252424 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:11.252449 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:11.254193 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:11.255204 (+ 1011us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:11.255208 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:11.255210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:11.255293 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:11.255649 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:11.255653 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:11.255655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:11.255896 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:11.256073 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:11.256486 (+ 413us) server_negotiation.cc:299] Negotiation successful
0808 18:27:11.256601 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260808 18:27:11.257692 15572 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ec28634d339405bad3ed43be0681772: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:11.258019 15491 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0a407ba2ea5402da1f6d793183db2ec [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:27:11.258544 15572 log.cc:824] T 00000000000000000000000000000000 P 3ec28634d339405bad3ed43be0681772: Log is configured to *not* fsync() on all Append() calls
I20260808 18:27:11.259255 15658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:11.245980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:44469 (local address 127.0.0.1:44658)
0808 18:27:11.246108 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:11.246122 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:11.246190 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:11.249985 (+ 3795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:11.249990 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:11.250165 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:11.250268 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:11.250289 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:11.251440 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:11.251444 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:11.252257 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:11.252267 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:11.252949 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:11.252953 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:11.253048 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:11.254302 (+ 1254us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:11.254312 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:11.255805 (+ 1493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:11.257803 (+ 1998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:11.257806 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:11.257807 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:11.258034 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:11.258329 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:11.258332 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:11.258337 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:11.258432 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:11.258811 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:11.258813 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:11.258897 (+ 84us) client_negotiation.cc:769] Sending connection context
0808 18:27:11.258977 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 18:27:11.259110 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 18:27:11.259859 15657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:11.246179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44658 (local address 127.23.72.188:44469)
0808 18:27:11.249752 (+ 3573us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:11.249759 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:11.249774 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:27:11.249810 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:11.249818 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:11.249863 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:11.250395 (+ 532us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:11.250479 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:11.251306 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:11.252395 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:11.252825 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:11.253252 (+ 427us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:11.255936 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:11.255941 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:11.255944 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:11.255965 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:11.257692 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:11.258141 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:11.258144 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:11.258145 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:11.258226 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:11.258525 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:11.258528 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:11.258529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:11.258716 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:11.259543 (+ 827us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:11.259617 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 18:27:11.259731 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3486}
I20260808 18:27:11.260788 23842 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:27:11.262948 15654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd0c81b88967477a8eb7f6bfd8f529f4: Bootstrap starting.
I20260808 18:27:11.260651 15661 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0a407ba2ea5402da1f6d793183db2ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0a407ba2ea5402da1f6d793183db2ec" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 43387 } } peers { permanent_uuid: "3ec28634d339405bad3ed43be0681772" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46719 } } peers { permanent_uuid: "dd0c81b88967477a8eb7f6bfd8f529f4" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 44469 } } }
I20260808 18:27:11.263945 15661 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0a407ba2ea5402da1f6d793183db2ec [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:27:11.264058 15572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ec28634d339405bad3ed43be0681772: No bootstrap required, opened a new log
I20260808 18:27:11.262101 15464 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "d0a407ba2ea5402da1f6d793183db2ec"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51882
I20260808 18:27:11.264453 15464 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0a407ba2ea5402da1f6d793183db2ec [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 18:27:11.264575 15464 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0a407ba2ea5402da1f6d793183db2ec [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:11.265609 15654 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd0c81b88967477a8eb7f6bfd8f529f4: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:11.265898 15464 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0a407ba2ea5402da1f6d793183db2ec [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0a407ba2ea5402da1f6d793183db2ec" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 43387 } } peers { permanent_uuid: "3ec28634d339405bad3ed43be0681772" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46719 } } peers { permanent_uuid: "dd0c81b88967477a8eb7f6bfd8f529f4" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 44469 } }
I20260808 18:27:11.266475 15654 log.cc:824] T 00000000000000000000000000000000 P dd0c81b88967477a8eb7f6bfd8f529f4: Log is configured to *not* fsync() on all Append() calls
I20260808 18:27:11.266820 15464 leader_election.cc:288] T 00000000000000000000000000000000 P d0a407ba2ea5402da1f6d793183db2ec [CANDIDATE]: Term 1 election: Requested vote from peers 3ec28634d339405bad3ed43be0681772 (127.23.72.189:46719), dd0c81b88967477a8eb7f6bfd8f529f4 (127.23.72.188:44469)
I20260808 18:27:11.268249 15572 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ec28634d339405bad3ed43be0681772 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0a407ba2ea5402da1f6d793183db2ec" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 43387 } } peers { permanent_uuid: "3ec28634d339405bad3ed43be0681772" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46719 } } peers { permanent_uuid: "dd0c81b88967477a8eb7f6bfd8f529f4" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 44469 } }
I20260808 18:27:11.268494 15572 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ec28634d339405bad3ed43be0681772 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ec28634d339405bad3ed43be0681772, State: Initialized, Role: FOLLOWER
W20260808 18:27:11.263327 15674 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d0a407ba2ea5402da1f6d793183db2ec: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:27:11.268831 15674 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:27:11.268836 15572 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ec28634d339405bad3ed43be0681772 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0a407ba2ea5402da1f6d793183db2ec" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 43387 } } peers { permanent_uuid: "3ec28634d339405bad3ed43be0681772" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46719 } } peers { permanent_uuid: "dd0c81b88967477a8eb7f6bfd8f529f4" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 44469 } }
I20260808 18:27:11.270947 15572 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ec28634d339405bad3ed43be0681772 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:27:11.271054 15654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd0c81b88967477a8eb7f6bfd8f529f4: No bootstrap required, opened a new log
I20260808 18:27:11.273507 15682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ec28634d339405bad3ed43be0681772 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0a407ba2ea5402da1f6d793183db2ec" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 43387 } } peers { permanent_uuid: "3ec28634d339405bad3ed43be0681772" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46719 } } peers { permanent_uuid: "dd0c81b88967477a8eb7f6bfd8f529f4" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 44469 } } }
I20260808 18:27:11.273636 15682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ec28634d339405bad3ed43be0681772 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:27:11.269954 15418 master_runner.cc:186] Error getting master registration for 127.23.72.189:46719: OK, instance_id { permanent_uuid: "3ec28634d339405bad3ed43be0681772" instance_seqno: 1786213630984042 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 18:27:11.274492 15418 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 18:27:11.273241 15543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d0a407ba2ea5402da1f6d793183db2ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3ec28634d339405bad3ed43be0681772"
I20260808 18:27:11.275095 15543 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ec28634d339405bad3ed43be0681772 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:11.275292 15654 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd0c81b88967477a8eb7f6bfd8f529f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0a407ba2ea5402da1f6d793183db2ec" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 43387 } } peers { permanent_uuid: "3ec28634d339405bad3ed43be0681772" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46719 } } peers { permanent_uuid: "dd0c81b88967477a8eb7f6bfd8f529f4" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 44469 } }
I20260808 18:27:11.275561 15654 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd0c81b88967477a8eb7f6bfd8f529f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd0c81b88967477a8eb7f6bfd8f529f4, State: Initialized, Role: FOLLOWER
I20260808 18:27:11.275880 15654 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd0c81b88967477a8eb7f6bfd8f529f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0a407ba2ea5402da1f6d793183db2ec" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 43387 } } peers { permanent_uuid: "3ec28634d339405bad3ed43be0681772" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46719 } } peers { permanent_uuid: "dd0c81b88967477a8eb7f6bfd8f529f4" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 44469 } }
W20260808 18:27:11.276968 15693 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3ec28634d339405bad3ed43be0681772: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:27:11.277043 15693 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:27:11.277285 15654 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd0c81b88967477a8eb7f6bfd8f529f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:27:11.277825 15624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d0a407ba2ea5402da1f6d793183db2ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "dd0c81b88967477a8eb7f6bfd8f529f4"
I20260808 18:27:11.278053 15624 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd0c81b88967477a8eb7f6bfd8f529f4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:11.277052 15543 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3ec28634d339405bad3ed43be0681772 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0a407ba2ea5402da1f6d793183db2ec in term 1.
I20260808 18:27:11.279443 15624 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dd0c81b88967477a8eb7f6bfd8f529f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0a407ba2ea5402da1f6d793183db2ec in term 1.
I20260808 18:27:11.279841 15439 leader_election.cc:302] T 00000000000000000000000000000000 P d0a407ba2ea5402da1f6d793183db2ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ec28634d339405bad3ed43be0681772, d0a407ba2ea5402da1f6d793183db2ec; no voters:
I20260808 18:27:11.280211 15661 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0a407ba2ea5402da1f6d793183db2ec [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:27:11.280470 15661 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0a407ba2ea5402da1f6d793183db2ec [term 1 LEADER]: Becoming Leader. State: Replica: d0a407ba2ea5402da1f6d793183db2ec, State: Running, Role: LEADER
I20260808 18:27:11.280821 15661 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0a407ba2ea5402da1f6d793183db2ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0a407ba2ea5402da1f6d793183db2ec" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 43387 } } peers { permanent_uuid: "3ec28634d339405bad3ed43be0681772" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46719 } } peers { permanent_uuid: "dd0c81b88967477a8eb7f6bfd8f529f4" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 44469 } }
I20260808 18:27:11.282717 15661 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0a407ba2ea5402da1f6d793183db2ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0a407ba2ea5402da1f6d793183db2ec. Latest consensus state: current_term: 1 leader_uuid: "d0a407ba2ea5402da1f6d793183db2ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0a407ba2ea5402da1f6d793183db2ec" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 43387 } } peers { permanent_uuid: "3ec28634d339405bad3ed43be0681772" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46719 } } peers { permanent_uuid: "dd0c81b88967477a8eb7f6bfd8f529f4" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 44469 } } }
I20260808 18:27:11.282867 15661 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0a407ba2ea5402da1f6d793183db2ec [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:11.284967 15500 master_runner.cc:186] Error getting master registration for 127.23.72.188:44469: OK, instance_id { permanent_uuid: "dd0c81b88967477a8eb7f6bfd8f529f4" instance_seqno: 1786213631207502 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 18:27:11.289912 15500 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 18:27:11.290853 15657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:11.275912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44672 (local address 127.23.72.188:44469)
0808 18:27:11.276047 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:11.276052 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:11.277014 (+ 962us) server_negotiation.cc:407] Connection header received
0808 18:27:11.277346 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:11.277351 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:11.277403 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:11.277456 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:11.280835 (+ 3379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:11.281630 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:11.282930 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:11.283270 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:11.283359 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:11.286538 (+ 3179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:11.286543 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:11.286546 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:11.286573 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:11.288520 (+ 1947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:11.289069 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:11.289072 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:11.289074 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:11.289165 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:11.289492 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:11.289496 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:11.289498 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:11.289736 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:11.289845 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:11.290587 (+ 742us) server_negotiation.cc:299] Negotiation successful
0808 18:27:11.290713 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 18:27:11.291440 15695 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd0c81b88967477a8eb7f6bfd8f529f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0a407ba2ea5402da1f6d793183db2ec" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 43387 } } peers { permanent_uuid: "3ec28634d339405bad3ed43be0681772" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46719 } } peers { permanent_uuid: "dd0c81b88967477a8eb7f6bfd8f529f4" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 44469 } } }
I20260808 18:27:11.291550 15695 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd0c81b88967477a8eb7f6bfd8f529f4 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 18:27:11.299157 15714 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dd0c81b88967477a8eb7f6bfd8f529f4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260808 18:27:11.299618 15712 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:27:11.300520 15497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:11.283844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51886 (local address 127.23.72.190:43387)
0808 18:27:11.283978 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:11.283983 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:11.291514 (+ 7531us) server_negotiation.cc:407] Connection header received
0808 18:27:11.291711 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:11.291716 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:11.291768 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:11.291821 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:11.293004 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:11.293502 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:11.294316 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:11.294685 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:11.294822 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:11.296490 (+ 1668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:11.296493 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:11.296496 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:11.296523 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:11.298307 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:11.298982 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:11.298986 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:11.298988 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:11.299096 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:11.299484 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:11.299487 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:11.299489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:11.299735 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:11.299868 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:11.300241 (+ 373us) server_negotiation.cc:299] Negotiation successful
0808 18:27:11.300375 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 18:27:11.302775 15712 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260808 18:27:11.305145 15714 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:27:11.312196 15576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:11.284298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50710 (local address 127.23.72.189:46719)
0808 18:27:11.284434 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:11.284440 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:11.301159 (+ 16719us) server_negotiation.cc:407] Connection header received
0808 18:27:11.301994 (+ 835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:11.301998 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:11.302052 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:11.302103 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:11.303838 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:11.304542 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:11.305611 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:11.305979 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:11.306063 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:11.308161 (+ 2098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:11.308166 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:11.308169 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:11.308206 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:11.310267 (+ 2061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:11.310833 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:11.310837 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:11.310839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:11.310927 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:11.311216 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:11.311219 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:11.311221 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:11.311464 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:11.311577 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:11.311919 (+ 342us) server_negotiation.cc:299] Negotiation successful
0808 18:27:11.312034 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 18:27:11.316119 15543 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3ec28634d339405bad3ed43be0681772 [term 1 FOLLOWER]: Refusing update from remote peer d0a407ba2ea5402da1f6d793183db2ec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:27:11.316119 15624 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dd0c81b88967477a8eb7f6bfd8f529f4 [term 1 FOLLOWER]: Refusing update from remote peer d0a407ba2ea5402da1f6d793183db2ec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:27:11.316795 15708 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d0a407ba2ea5402da1f6d793183db2ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd0c81b88967477a8eb7f6bfd8f529f4" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 44469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:27:11.317004 15661 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d0a407ba2ea5402da1f6d793183db2ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ec28634d339405bad3ed43be0681772" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:27:11.320578 15682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ec28634d339405bad3ed43be0681772 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0a407ba2ea5402da1f6d793183db2ec. Latest consensus state: current_term: 1 leader_uuid: "d0a407ba2ea5402da1f6d793183db2ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0a407ba2ea5402da1f6d793183db2ec" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 43387 } } peers { permanent_uuid: "3ec28634d339405bad3ed43be0681772" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46719 } } peers { permanent_uuid: "dd0c81b88967477a8eb7f6bfd8f529f4" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 44469 } } }
I20260808 18:27:11.320698 15682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ec28634d339405bad3ed43be0681772 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:27:11.320825 15695 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd0c81b88967477a8eb7f6bfd8f529f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0a407ba2ea5402da1f6d793183db2ec. Latest consensus state: current_term: 1 leader_uuid: "d0a407ba2ea5402da1f6d793183db2ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0a407ba2ea5402da1f6d793183db2ec" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 43387 } } peers { permanent_uuid: "3ec28634d339405bad3ed43be0681772" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46719 } } peers { permanent_uuid: "dd0c81b88967477a8eb7f6bfd8f529f4" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 44469 } } }
I20260808 18:27:11.320933 15695 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd0c81b88967477a8eb7f6bfd8f529f4 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:27:11.326263 15708 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0a407ba2ea5402da1f6d793183db2ec [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d0a407ba2ea5402da1f6d793183db2ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0a407ba2ea5402da1f6d793183db2ec" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 43387 } } peers { permanent_uuid: "3ec28634d339405bad3ed43be0681772" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46719 } } peers { permanent_uuid: "dd0c81b88967477a8eb7f6bfd8f529f4" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 44469 } } }
I20260808 18:27:11.326402 15708 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0a407ba2ea5402da1f6d793183db2ec [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:11.326531 15717 mvcc.cc:205] Tried to move back new op lower bound from 7316331033863487488 to 7316331033730019328. Current Snapshot: MvccSnapshot[applied={T|T < 7316331033863487488}]
I20260808 18:27:11.327477 15682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ec28634d339405bad3ed43be0681772 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d0a407ba2ea5402da1f6d793183db2ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0a407ba2ea5402da1f6d793183db2ec" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 43387 } } peers { permanent_uuid: "3ec28634d339405bad3ed43be0681772" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46719 } } peers { permanent_uuid: "dd0c81b88967477a8eb7f6bfd8f529f4" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 44469 } } }
I20260808 18:27:11.327593 15682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ec28634d339405bad3ed43be0681772 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:27:11.328153 15719 mvcc.cc:205] Tried to move back new op lower bound from 7316331033863487488 to 7316331033730019328. Current Snapshot: MvccSnapshot[applied={T|T < 7316331033863487488}]
I20260808 18:27:11.331945 15661 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0a407ba2ea5402da1f6d793183db2ec [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d0a407ba2ea5402da1f6d793183db2ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0a407ba2ea5402da1f6d793183db2ec" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 43387 } } peers { permanent_uuid: "3ec28634d339405bad3ed43be0681772" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46719 } } peers { permanent_uuid: "dd0c81b88967477a8eb7f6bfd8f529f4" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 44469 } } }
I20260808 18:27:11.332068 15661 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0a407ba2ea5402da1f6d793183db2ec [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:11.332779 15695 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd0c81b88967477a8eb7f6bfd8f529f4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d0a407ba2ea5402da1f6d793183db2ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0a407ba2ea5402da1f6d793183db2ec" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 43387 } } peers { permanent_uuid: "3ec28634d339405bad3ed43be0681772" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46719 } } peers { permanent_uuid: "dd0c81b88967477a8eb7f6bfd8f529f4" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 44469 } } }
I20260808 18:27:11.332911 15695 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd0c81b88967477a8eb7f6bfd8f529f4 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:27:11.340291 15712 catalog_manager.cc:1347] Generated new cluster ID: 2b0eecc23693483cb18f7fd1c027280c
I20260808 18:27:11.340369 15712 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:27:11.369315 15712 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:27:11.370615 15712 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:27:11.385921 15712 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0a407ba2ea5402da1f6d793183db2ec: Generated new TSK 0
I20260808 18:27:11.386808 15712 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 18:27:11.422709 15576 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:27:11.420016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50724 (local address 127.23.72.189:46719)
0808 18:27:11.420157 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:11.420162 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:11.421579 (+ 1417us) server_negotiation.cc:407] Connection header received
0808 18:27:11.421796 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:11.421800 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:11.421854 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:11.421908 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:11.422579 (+ 671us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50724: BlockingRecv error: recv got EOF from 127.0.0.1:50724 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260808 18:27:11.424432 15497 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:27:11.420894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51900 (local address 127.23.72.190:43387)
0808 18:27:11.421025 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:11.421030 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:11.423288 (+ 2258us) server_negotiation.cc:407] Connection header received
0808 18:27:11.423573 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:11.423577 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:11.423637 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:11.423693 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:11.424289 (+ 596us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51900: BlockingRecv error: recv got EOF from 127.0.0.1:51900 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260808 18:27:11.425940 15657 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:27:11.425017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44676 (local address 127.23.72.188:44469)
0808 18:27:11.425150 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:11.425154 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:11.425171 (+ 17us) server_negotiation.cc:407] Connection header received
0808 18:27:11.425217 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:11.425220 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:11.425267 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:11.425377 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:11.425813 (+ 436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44676: BlockingRecv error: recv got EOF from 127.0.0.1:44676 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260808 18:27:11.430330 15576 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:27:11.428712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50736 (local address 127.23.72.189:46719)
0808 18:27:11.428825 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:11.428829 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:11.429303 (+ 474us) server_negotiation.cc:407] Connection header received
0808 18:27:11.429529 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:11.429532 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:11.429612 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:11.429666 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:11.430218 (+ 552us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50736: BlockingRecv error: recv got EOF from 127.0.0.1:50736 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260808 18:27:11.435715 15576 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:27:11.433927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50752 (local address 127.23.72.189:46719)
0808 18:27:11.434057 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:11.434064 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:11.434699 (+ 635us) server_negotiation.cc:407] Connection header received
0808 18:27:11.434887 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:11.434890 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:11.434942 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:11.435007 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:11.435589 (+ 582us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50752: BlockingRecv error: recv got EOF from 127.0.0.1:50752 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260808 18:27:11.441529 15576 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:27:11.439906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50756 (local address 127.23.72.189:46719)
0808 18:27:11.440026 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:11.440031 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:11.440588 (+ 557us) server_negotiation.cc:407] Connection header received
0808 18:27:11.440781 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:11.440785 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:11.440836 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:11.440888 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:11.441416 (+ 528us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50756: BlockingRecv error: recv got EOF from 127.0.0.1:50756 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260808 18:27:11.449559 15576 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:27:11.447710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50760 (local address 127.23.72.189:46719)
0808 18:27:11.447869 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:11.447874 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:11.448574 (+ 700us) server_negotiation.cc:407] Connection header received
0808 18:27:11.448776 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:11.448779 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:11.448834 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:11.448890 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:11.449449 (+ 559us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50760: BlockingRecv error: recv got EOF from 127.0.0.1:50760 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260808 18:27:11.462095 15576 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:27:11.460318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50768 (local address 127.23.72.189:46719)
0808 18:27:11.460440 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:11.460446 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:11.461044 (+ 598us) server_negotiation.cc:407] Connection header received
0808 18:27:11.461285 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:11.461289 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:11.461346 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:11.461404 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:11.461983 (+ 579us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50768: BlockingRecv error: recv got EOF from 127.0.0.1:50768 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260808 18:27:11.483709 15576 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:27:11.481742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50778 (local address 127.23.72.189:46719)
0808 18:27:11.481876 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:11.481883 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:11.482590 (+ 707us) server_negotiation.cc:407] Connection header received
0808 18:27:11.482801 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:11.482805 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:11.482861 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:11.482917 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:11.483582 (+ 665us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50778: BlockingRecv error: recv got EOF from 127.0.0.1:50778 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260808 18:27:11.519855 15576 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:27:11.518501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50780 (local address 127.23.72.189:46719)
0808 18:27:11.518690 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:11.518695 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:11.518856 (+ 161us) server_negotiation.cc:407] Connection header received
0808 18:27:11.518992 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:11.518996 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:11.519057 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:11.519163 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:11.519732 (+ 569us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50780: BlockingRecv error: recv got EOF from 127.0.0.1:50780 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260808 18:27:11.587173 15576 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:27:11.586071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50790 (local address 127.23.72.189:46719)
0808 18:27:11.586220 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:11.586223 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:11.586382 (+ 159us) server_negotiation.cc:407] Connection header received
0808 18:27:11.586490 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:11.586493 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:11.586544 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:11.586629 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:11.587072 (+ 443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50790: BlockingRecv error: recv got EOF from 127.0.0.1:50790 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260808 18:27:11.718537 15576 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:27:11.717382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50792 (local address 127.23.72.189:46719)
0808 18:27:11.717587 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:11.717591 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:11.717624 (+ 33us) server_negotiation.cc:407] Connection header received
0808 18:27:11.717766 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:11.717768 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:11.717815 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:11.717939 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:11.718401 (+ 462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50792: BlockingRecv error: recv got EOF from 127.0.0.1:50792 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
W20260808 18:27:11.977993 15576 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:27:11.976788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50802 (local address 127.23.72.189:46719)
0808 18:27:11.976954 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:11.976959 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:11.977057 (+ 98us) server_negotiation.cc:407] Connection header received
0808 18:27:11.977250 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:11.977253 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:11.977303 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:11.977378 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:11.977861 (+ 483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50802: BlockingRecv error: recv got EOF from 127.0.0.1:50802 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 18:27:12.278537 15693 catalog_manager.cc:1259] Loaded cluster ID: 2b0eecc23693483cb18f7fd1c027280c
I20260808 18:27:12.278622 15693 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3ec28634d339405bad3ed43be0681772: loading cluster ID for follower catalog manager: success
I20260808 18:27:12.280403 15693 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3ec28634d339405bad3ed43be0681772: acquiring CA information for follower catalog manager: success
I20260808 18:27:12.281564 15693 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3ec28634d339405bad3ed43be0681772: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 18:27:12.306550 15714 catalog_manager.cc:1259] Loaded cluster ID: 2b0eecc23693483cb18f7fd1c027280c
I20260808 18:27:12.306689 15714 catalog_manager.cc:1552] T 00000000000000000000000000000000 P dd0c81b88967477a8eb7f6bfd8f529f4: loading cluster ID for follower catalog manager: success
I20260808 18:27:12.308481 15714 catalog_manager.cc:1574] T 00000000000000000000000000000000 P dd0c81b88967477a8eb7f6bfd8f529f4: acquiring CA information for follower catalog manager: success
I20260808 18:27:12.309706 15714 catalog_manager.cc:1602] T 00000000000000000000000000000000 P dd0c81b88967477a8eb7f6bfd8f529f4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 18:27:12.495772 15797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:12.492142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50816 (local address 127.23.72.189:46719)
0808 18:27:12.492407 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:12.492411 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:12.492431 (+ 20us) server_negotiation.cc:407] Connection header received
0808 18:27:12.492578 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:12.492580 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:12.492707 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:12.492809 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:27:12.493041 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:12.493549 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:12.494338 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:12.494549 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:12.494623 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:12.494896 (+ 273us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:27:12.495325 (+ 429us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:27:12.495415 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:12.495530 (+ 115us) server_negotiation.cc:299] Negotiation successful
0808 18:27:12.495576 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":74,"threads_started":1}
I20260808 18:27:12.502115 15804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:12.498791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44690 (local address 127.23.72.188:44469)
0808 18:27:12.499023 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:12.499026 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:12.499038 (+ 12us) server_negotiation.cc:407] Connection header received
0808 18:27:12.499111 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:12.499113 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:12.499217 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:12.499308 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:27:12.499545 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:12.500052 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:12.500788 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:12.501024 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:12.501103 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:12.501333 (+ 230us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:27:12.501693 (+ 360us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:27:12.501781 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:12.501872 (+ 91us) server_negotiation.cc:299] Negotiation successful
0808 18:27:12.501919 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":79,"threads_started":1}
I20260808 18:27:12.509617 15797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:12.505230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50828 (local address 127.23.72.189:46719)
0808 18:27:12.505352 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:12.505356 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:12.505914 (+ 558us) server_negotiation.cc:407] Connection header received
0808 18:27:12.506062 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:12.506066 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:12.506120 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:12.506172 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:27:12.506503 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:12.506985 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:12.508197 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:12.508549 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:12.508617 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:12.509094 (+ 477us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:27:12.509200 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:27:12.509272 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:12.509427 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 18:27:12.509479 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 18:27:12.509716 15804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:12.505433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44692 (local address 127.23.72.188:44469)
0808 18:27:12.505538 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:12.505542 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:12.505554 (+ 12us) server_negotiation.cc:407] Connection header received
0808 18:27:12.505598 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:12.505601 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:12.505655 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:12.505728 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:27:12.505990 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:12.506741 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:12.508058 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:12.508368 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:12.508436 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:12.509046 (+ 610us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:27:12.509156 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:27:12.509244 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:12.509491 (+ 247us) server_negotiation.cc:299] Negotiation successful
0808 18:27:12.509557 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 18:27:12.510589 15811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:12.505006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51908 (local address 127.23.72.190:43387)
0808 18:27:12.505269 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:12.505274 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:12.505481 (+ 207us) server_negotiation.cc:407] Connection header received
0808 18:27:12.505729 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:12.505733 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:12.505892 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:12.506009 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:27:12.506226 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:12.506982 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:12.507897 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:12.508217 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:12.508337 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:12.508654 (+ 317us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:27:12.510117 (+ 1463us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:27:12.510203 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:12.510368 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 18:27:12.510432 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":76,"threads_started":1}
I20260808 18:27:12.513612 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 15418
I20260808 18:27:12.522444 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 15500
I20260808 18:27:12.529534 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 15579
2026-08-08T18:27:12Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3156 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:27:12 dist-test-slave-5k54 krb5kdc[15815](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:27:12 dist-test-slave-5k54 krb5kdc[15815](info): set up 2 sockets
Aug 08 18:27:12 dist-test-slave-5k54 krb5kdc[15815](info): commencing operation
krb5kdc: starting...
W20260808 18:27:13.608569 23842 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/krb5kdc/test-user.keytab.
Aug 08 18:27:13 dist-test-slave-5k54 krb5kdc[15815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:27:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:27:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.72.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
Entry for principal kudu/127.23.72.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab.
I20260808 18:27:13.734721 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:40887
--webserver_interface=127.23.72.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:34609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:40887,127.23.72.189:46103,127.23.72.188:46559
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:13.849046 15828 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 18:27:13.849396 15828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:13.849468 15828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:13.849553 15828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:13.853560 15828 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:27:13.853652 15828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:13.853688 15828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:13.853726 15828 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:27:13.853760 15828 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:27:13.859033 15828 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:34609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:40887,127.23.72.189:46103,127.23.72.188:46559
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.190.keytab
--principal=kudu/127.23.72.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.190:40887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.72.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15828
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:13.860191 15828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:13.861294 15828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 18:27:13.870079 15828 server_base.cc:1034] running on GCE node
W20260808 18:27:13.869992 15836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:13.869969 15834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:13.869980 15833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:13.870858 15828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:13.871991 15828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:13.873155 15828 hybrid_clock.cc:648] HybridClock initialized: now 1786213633873133 us; error 39 us; skew 500 ppm
Aug 08 18:27:13 dist-test-slave-5k54 krb5kdc[15815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213633, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:13.876041 15828 init.cc:377] Logged in from keytab as kudu/127.23.72.190@KRBTEST.COM (short username kudu)
I20260808 18:27:13.876901 15828 webserver.cc:460] Webserver started at http://127.23.72.190:43595/ using document root <none> and password file <none>
I20260808 18:27:13.877491 15828 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:13.877552 15828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:13.877770 15828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:13.879885 15828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-0/data/instance:
uuid: "4b2e9b5598014afd898b7854e73f5172"
format_stamp: "Formatted at 2026-08-08 18:27:13 on dist-test-slave-5k54"
server_key: "9de819ab3124dc3c89bcb78a1491c526"
server_key_iv: "9aaa84f5a63e7e0e2d1de367f6dba948"
server_key_version: "encryptionkey@0"
I20260808 18:27:13.880473 15828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-0/wal/instance:
uuid: "4b2e9b5598014afd898b7854e73f5172"
format_stamp: "Formatted at 2026-08-08 18:27:13 on dist-test-slave-5k54"
server_key: "9de819ab3124dc3c89bcb78a1491c526"
server_key_iv: "9aaa84f5a63e7e0e2d1de367f6dba948"
server_key_version: "encryptionkey@0"
I20260808 18:27:13.884207 15828 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 18:27:13.886662 15843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:13.888193 15828 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 18:27:13.888332 15828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-0/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-0/wal
uuid: "4b2e9b5598014afd898b7854e73f5172"
format_stamp: "Formatted at 2026-08-08 18:27:13 on dist-test-slave-5k54"
server_key: "9de819ab3124dc3c89bcb78a1491c526"
server_key_iv: "9aaa84f5a63e7e0e2d1de367f6dba948"
server_key_version: "encryptionkey@0"
I20260808 18:27:13.888445 15828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:13.906215 15828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:13.909386 15828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:13.909595 15828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:13.917213 15828 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.190:40887
I20260808 18:27:13.917520 15902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.190:40887 every 8 connection(s)
I20260808 18:27:13.918232 15828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-0/data/info.pb
I20260808 18:27:13.921298 15903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:13.921545 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 15828
I20260808 18:27:13.921671 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-0/wal/instance
I20260808 18:27:13.922041 23842 external_mini_cluster.cc:1428] Setting key b7c233811b0ef616a3969da03ebbef0c
I20260808 18:27:13.925997 15903 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40887 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.23.72.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.189@KRBTEST.COM" created.
Aug 08 18:27:13 dist-test-slave-5k54 krb5kdc[15815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213633, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:27:13.942634 15907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:13.928412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:40887 (local address 127.0.0.1:44008)
0808 18:27:13.929017 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:13.929063 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:13.929916 (+ 853us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:13.930749 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:13.930754 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:13.931144 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:13.931400 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:13.931412 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:13.932743 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:13.932746 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:13.933496 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:13.933505 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:13.934384 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:13.934391 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:13.934561 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:13.935190 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:13.935216 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:13.936924 (+ 1708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:13.939399 (+ 2475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:13.939409 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:13.939417 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:13.939749 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:13.940200 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:13.940202 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:13.940203 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:13.940327 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:13.941163 (+ 836us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:13.941168 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:13.941297 (+ 129us) client_negotiation.cc:769] Sending connection context
0808 18:27:13.941451 (+ 154us) client_negotiation.cc:240] Negotiation successful
0808 18:27:13.941726 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"spinlock_wait_cycles":13696,"thread_start_us":158,"threads_started":1}
I20260808 18:27:13.942637 15906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:13.928412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44008 (local address 127.23.72.190:40887)
0808 18:27:13.928895 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:13.928923 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:13.929189 (+ 266us) server_negotiation.cc:407] Connection header received
0808 18:27:13.930252 (+ 1063us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:13.930260 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:13.930532 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:13.930737 (+ 205us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:13.931617 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:13.932597 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:13.933601 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:13.934191 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:13.934313 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:13.937078 (+ 2765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:13.937088 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:13.937103 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:13.937136 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:13.939272 (+ 2136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:13.939914 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:13.939924 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:13.939928 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:13.940083 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:13.940445 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:13.940450 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:13.940455 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:13.941016 (+ 561us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:13.941168 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:13.941591 (+ 423us) server_negotiation.cc:299] Negotiation successful
0808 18:27:13.941836 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":316,"thread_start_us":134,"threads_started":1}
I20260808 18:27:13.945149 15903 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46103 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:27:13.945964 15907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:13.945696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.189:46103 (local address 127.0.0.1:58694)
0808 18:27:13.945829 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:13.945905 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.189:46103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260808 18:27:13.946220 15845 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.72.189:46103: connect: Connection refused (error 111)
I20260808 18:27:13.947131 15907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:13.946880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.189:46103 (local address 127.0.0.1:58706)
0808 18:27:13.947023 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:13.947081 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.189:46103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260808 18:27:13.947296 15903 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.72.189:46103: Network error: Client connection negotiation failed: client connection to 127.23.72.189:46103: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.189.keytab.
Entry for principal kudu/127.23.72.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.189.keytab.
I20260808 18:27:13.951331 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.189:46103
--webserver_interface=127.23.72.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.189.keytab
--principal=kudu/127.23.72.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:34609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:40887,127.23.72.189:46103,127.23.72.188:46559
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:27:13.996618 15903 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46103 } attempt: 1
I20260808 18:27:13.997541 15907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:13.997192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.189:46103 (local address 127.0.0.1:58714)
0808 18:27:13.997368 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:13.997445 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.189:46103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260808 18:27:13.998315 15907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:13.998078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.189:46103 (local address 127.0.0.1:58728)
0808 18:27:13.998188 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:13.998243 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.189:46103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 18:27:13.998474 15903 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.72.189:46103: Network error: Client connection negotiation failed: client connection to 127.23.72.189:46103: connect: Connection refused (error 111)
W20260808 18:27:14.063676 15911 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 18:27:14.063988 15911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:14.064069 15911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:14.064149 15911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 18:27:14.066639 15903 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46103 } attempt: 2
I20260808 18:27:14.067524 15907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:14.067148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.189:46103 (local address 127.0.0.1:58734)
0808 18:27:14.067327 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:14.067400 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.189:46103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260808 18:27:14.068257 15907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:14.068054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.189:46103 (local address 127.0.0.1:58736)
0808 18:27:14.068153 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:14.068202 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.189:46103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 18:27:14.068470 15903 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.72.189:46103: Network error: Client connection negotiation failed: client connection to 127.23.72.189:46103: connect: Connection refused (error 111)
W20260808 18:27:14.068629 15911 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:27:14.068699 15911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:14.068743 15911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:14.068780 15911 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:27:14.068812 15911 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:27:14.073958 15911 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:34609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:40887,127.23.72.189:46103,127.23.72.188:46559
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.189.keytab
--principal=kudu/127.23.72.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.189:46103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.23.72.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:14.074958 15911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:14.076011 15911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 18:27:14.084411 15911 server_base.cc:1034] running on GCE node
W20260808 18:27:14.084388 15919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:14.084364 15917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:14.084348 15916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:14.085165 15911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:14.086261 15911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:14.087482 15911 hybrid_clock.cc:648] HybridClock initialized: now 1786213634087466 us; error 25 us; skew 500 ppm
Aug 08 18:27:14 dist-test-slave-5k54 krb5kdc[15815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213634, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:14.090304 15911 init.cc:377] Logged in from keytab as kudu/127.23.72.189@KRBTEST.COM (short username kudu)
I20260808 18:27:14.091234 15911 webserver.cc:460] Webserver started at http://127.23.72.189:40889/ using document root <none> and password file <none>
I20260808 18:27:14.091869 15911 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:14.091961 15911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:14.092170 15911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:14.094200 15911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-1/data/instance:
uuid: "c4b0579168b34515b33db59f918997da"
format_stamp: "Formatted at 2026-08-08 18:27:14 on dist-test-slave-5k54"
server_key: "50653db2463be08b868574e0822d92d5"
server_key_iv: "0055d482d69f006225c4cc19426507c9"
server_key_version: "encryptionkey@0"
I20260808 18:27:14.094763 15911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-1/wal/instance:
uuid: "c4b0579168b34515b33db59f918997da"
format_stamp: "Formatted at 2026-08-08 18:27:14 on dist-test-slave-5k54"
server_key: "50653db2463be08b868574e0822d92d5"
server_key_iv: "0055d482d69f006225c4cc19426507c9"
server_key_version: "encryptionkey@0"
I20260808 18:27:14.098387 15911 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 18:27:14.100708 15926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:14.101956 15911 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:27:14.102123 15911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-1/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-1/wal
uuid: "c4b0579168b34515b33db59f918997da"
format_stamp: "Formatted at 2026-08-08 18:27:14 on dist-test-slave-5k54"
server_key: "50653db2463be08b868574e0822d92d5"
server_key_iv: "0055d482d69f006225c4cc19426507c9"
server_key_version: "encryptionkey@0"
I20260808 18:27:14.102239 15911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:14.129827 15911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:14.133281 15911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:14.133505 15911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:14.140583 15911 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.189:46103
I20260808 18:27:14.140858 15983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.189:46103 every 8 connection(s)
I20260808 18:27:14.141573 15911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-1/data/info.pb
I20260808 18:27:14.144299 15984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:14.148030 15984 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40887 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:27:14.148903 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 15911
I20260808 18:27:14.149053 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-1/wal/instance
I20260808 18:27:14.149328 23842 external_mini_cluster.cc:1428] Setting key 7a4f17986c11caa1acaf5ecaa807b8ff
Aug 08 18:27:14 dist-test-slave-5k54 krb5kdc[15815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213634, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.189@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.72.188@KRBTEST.COM; defaulting to no policy
I20260808 18:27:14.159622 15903 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46103 } attempt: 3
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.72.188@KRBTEST.COM" created.
I20260808 18:27:14.163594 15906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:14.150013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44020 (local address 127.23.72.190:40887)
0808 18:27:14.150149 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:14.150152 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:14.151117 (+ 965us) server_negotiation.cc:407] Connection header received
0808 18:27:14.152422 (+ 1305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:14.152425 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:14.152479 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:14.152557 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:14.153875 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:14.154406 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:14.155574 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:14.155816 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:14.155942 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:14.158306 (+ 2364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:14.158310 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:14.158312 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:14.158335 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:14.161074 (+ 2739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:14.161704 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:14.161708 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:14.161711 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:14.161823 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:14.162266 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:14.162270 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:14.162272 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:14.162490 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:14.162598 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:14.163272 (+ 674us) server_negotiation.cc:299] Negotiation successful
0808 18:27:14.163410 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 18:27:14.164785 15986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:14.150477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:40887 (local address 127.0.0.1:44020)
0808 18:27:14.150967 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:14.151009 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:14.152173 (+ 1164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:14.152774 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:14.152785 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:14.153262 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:14.153720 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:14.153732 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:14.154539 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:14.154548 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:14.155389 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:14.155397 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:14.155970 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:14.155974 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:14.156149 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:14.156765 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:14.156780 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:14.158137 (+ 1357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:14.161207 (+ 3070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:14.161213 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:14.161227 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:14.161592 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:14.161956 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:14.161961 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:14.161966 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:14.162152 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:14.162613 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:14.162622 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:14.162969 (+ 347us) client_negotiation.cc:769] Sending connection context
0808 18:27:14.163197 (+ 228us) client_negotiation.cc:240] Negotiation successful
0808 18:27:14.163616 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":148,"threads_started":1}
Aug 08 18:27:14 dist-test-slave-5k54 krb5kdc[15815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213633, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for kudu/127.23.72.189@KRBTEST.COM
I20260808 18:27:14.166361 15984 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46103 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:27:14.170924 15907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:14.160054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.189:46103 (local address 127.0.0.1:58740)
0808 18:27:14.160220 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:14.160231 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:14.160315 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:14.160901 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:14.160904 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:14.161138 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:14.161297 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:14.161309 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:14.162677 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:14.162681 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:14.163257 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:14.163265 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:14.164093 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:14.164096 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:14.164209 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:14.164676 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:14.164690 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:14.166158 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:14.169153 (+ 2995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:14.169156 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:14.169158 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:14.169416 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:14.169727 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:14.169731 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:14.169733 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:14.169820 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:14.170396 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:14.170399 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:14.170496 (+ 97us) client_negotiation.cc:769] Sending connection context
0808 18:27:14.170614 (+ 118us) client_negotiation.cc:240] Negotiation successful
0808 18:27:14.170752 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108}
I20260808 18:27:14.171041 15988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:14.160173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58740 (local address 127.23.72.189:46103)
0808 18:27:14.160498 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:14.160512 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:14.160536 (+ 24us) server_negotiation.cc:407] Connection header received
0808 18:27:14.160580 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:14.160584 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:14.160779 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:14.160916 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:14.161440 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:14.162546 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:14.163388 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:14.163959 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:14.164097 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:14.166312 (+ 2215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:14.166319 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:14.166328 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:14.166373 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:14.169031 (+ 2658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:14.169540 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:14.169546 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:14.169547 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:14.169633 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:14.169959 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:14.169962 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:14.169963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:14.170280 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:14.170436 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:14.170725 (+ 289us) server_negotiation.cc:299] Negotiation successful
0808 18:27:14.170857 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"spinlock_wait_cycles":896,"thread_start_us":93,"threads_started":1}
I20260808 18:27:14.172858 15903 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.72.188" port: 46559 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 18:27:14 dist-test-slave-5k54 krb5kdc[15815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213634, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.189@KRBTEST.COM for kudu/127.23.72.189@KRBTEST.COM
I20260808 18:27:14.173646 15907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:14.173359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:46559 (local address 127.0.0.1:59544)
0808 18:27:14.173512 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:14.173568 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.188:46559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 18:27:14.174459 15907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:14.174151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:46559 (local address 127.0.0.1:59548)
0808 18:27:14.174308 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:14.174376 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.188:46559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260808 18:27:14.174672 15903 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.72.188:46559: Network error: Client connection negotiation failed: client connection to 127.23.72.188:46559: connect: Connection refused (error 111)
I20260808 18:27:14.177281 15986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:14.166878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.189:46103 (local address 127.0.0.1:58756)
0808 18:27:14.166984 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:14.166998 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:14.167067 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:14.167588 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:14.167594 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:14.167794 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:14.167938 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:14.167948 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:14.168667 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:14.168673 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:14.169262 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:14.169271 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:14.169924 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:14.169927 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:14.170106 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:14.171207 (+ 1101us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:14.171221 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:14.173945 (+ 2724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:14.175864 (+ 1919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:14.175867 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:14.175869 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:14.176055 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:14.176345 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:14.176348 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:14.176349 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:14.176431 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:14.176827 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:14.176829 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:14.176905 (+ 76us) client_negotiation.cc:769] Sending connection context
0808 18:27:14.176986 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 18:27:14.177078 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":54}
I20260808 18:27:14.177297 15990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:14.167068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58756 (local address 127.23.72.189:46103)
0808 18:27:14.167346 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:14.167355 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:14.167367 (+ 12us) server_negotiation.cc:407] Connection header received
0808 18:27:14.167402 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:14.167406 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:14.167471 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:14.167554 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:14.168067 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:14.168555 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:14.169392 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:14.169803 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:14.169920 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:14.174091 (+ 4171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:14.174094 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:14.174097 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:14.174126 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:14.175747 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:14.176150 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:14.176153 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:14.176155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:14.176250 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:14.176511 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:14.176513 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:14.176514 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:14.176733 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:14.176827 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:14.177022 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 18:27:14.177154 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":208,"thread_start_us":96,"threads_started":1}
I20260808 18:27:14.177994 15984 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.72.188" port: 46559 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.72.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.188.keytab.
Entry for principal kudu/127.23.72.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.188.keytab.
I20260808 18:27:14.178644 15986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:14.178386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:46559 (local address 127.0.0.1:59550)
0808 18:27:14.178502 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:14.178584 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.188:46559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 18:27:14.178850 15928 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.72.188:46559: connect: Connection refused (error 111)
I20260808 18:27:14.179066 23842 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVyYox2/build/debug/bin/kudu
/tmp/dist-test-taskVyYox2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.188:46559
--webserver_interface=127.23.72.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.188.keytab
--principal=kudu/127.23.72.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.72.148:34609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.72.190:40887,127.23.72.189:46103,127.23.72.188:46559
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:27:14.179809 15986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:14.179533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:46559 (local address 127.0.0.1:59564)
0808 18:27:14.179670 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:14.179733 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.188:46559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 18:27:14.179966 15984 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.72.188:46559: Network error: Client connection negotiation failed: client connection to 127.23.72.188:46559: connect: Connection refused (error 111)
I20260808 18:27:14.205883 15903 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.72.188" port: 46559 } attempt: 1
I20260808 18:27:14.206674 15907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:14.206391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:46559 (local address 127.0.0.1:59568)
0808 18:27:14.206533 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:14.206597 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.188:46559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260808 18:27:14.207469 15907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:14.207205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:46559 (local address 127.0.0.1:59584)
0808 18:27:14.207315 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:14.207366 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.188:46559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 18:27:14.207628 15903 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.72.188:46559: Network error: Client connection negotiation failed: client connection to 127.23.72.188:46559: connect: Connection refused (error 111)
I20260808 18:27:14.229215 15984 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.72.188" port: 46559 } attempt: 1
I20260808 18:27:14.230093 15986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:14.229809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:46559 (local address 127.0.0.1:59588)
0808 18:27:14.229957 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:14.230015 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.188:46559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260808 18:27:14.230786 15986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:14.230586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:46559 (local address 127.0.0.1:59594)
0808 18:27:14.230690 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:14.230739 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.188:46559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 18:27:14.230908 15984 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.72.188:46559: Network error: Client connection negotiation failed: client connection to 127.23.72.188:46559: connect: Connection refused (error 111)
I20260808 18:27:14.282785 15903 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.72.188" port: 46559 } attempt: 2
I20260808 18:27:14.283735 15907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:14.283398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:46559 (local address 127.0.0.1:59608)
0808 18:27:14.283592 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:14.283653 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.188:46559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260808 18:27:14.284489 15907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:14.284276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:46559 (local address 127.0.0.1:59620)
0808 18:27:14.284383 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:14.284435 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.188:46559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 18:27:14.284615 15903 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.72.188:46559: Network error: Client connection negotiation failed: client connection to 127.23.72.188:46559: connect: Connection refused (error 111)
W20260808 18:27:14.291774 15992 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 18:27:14.292086 15992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:14.292146 15992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:14.292224 15992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:14.296325 15992 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:27:14.296396 15992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:14.296443 15992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:14.296476 15992 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:27:14.296530 15992 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:27:14.299059 15984 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.72.188" port: 46559 } attempt: 2
I20260808 18:27:14.299801 15986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:14.299540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:46559 (local address 127.0.0.1:59628)
0808 18:27:14.299665 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:14.299731 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.188:46559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 18:27:14.300508 15986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:14.300298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:46559 (local address 127.0.0.1:59630)
0808 18:27:14.300402 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:14.300454 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.72.188:46559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 18:27:14.300627 15984 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.72.188:46559: Network error: Client connection negotiation failed: client connection to 127.23.72.188:46559: connect: Connection refused (error 111)
I20260808 18:27:14.302026 15992 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.72.148:34609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.23.72.190:40887,127.23.72.189:46103,127.23.72.188:46559
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/krb5kdc/kudu_127.23.72.188.keytab
--principal=kudu/127.23.72.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.72.188:46559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.23.72.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:14.303047 15992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:14.304065 15992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:14.312453 15999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:14.312480 15998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:14.312453 16001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:14.312714 15992 server_base.cc:1034] running on GCE node
I20260808 18:27:14.313335 15992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:14.314400 15992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:14.315587 15992 hybrid_clock.cc:648] HybridClock initialized: now 1786213634315561 us; error 47 us; skew 500 ppm
Aug 08 18:27:14 dist-test-slave-5k54 krb5kdc[15815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213634, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:14.318233 15992 init.cc:377] Logged in from keytab as kudu/127.23.72.188@KRBTEST.COM (short username kudu)
I20260808 18:27:14.319036 15992 webserver.cc:460] Webserver started at http://127.23.72.188:34243/ using document root <none> and password file <none>
I20260808 18:27:14.319612 15992 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:14.319674 15992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:14.319835 15992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:14.321973 15992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-2/data/instance:
uuid: "ed9e33a81d66483dafc2dcb7572b3e6e"
format_stamp: "Formatted at 2026-08-08 18:27:14 on dist-test-slave-5k54"
server_key: "e2b5fb0821433fbb5e96e1ab7a879bce"
server_key_iv: "573e2f63733b3158ee63e74e4edfa1c1"
server_key_version: "encryptionkey@0"
I20260808 18:27:14.322738 15992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-2/wal/instance:
uuid: "ed9e33a81d66483dafc2dcb7572b3e6e"
format_stamp: "Formatted at 2026-08-08 18:27:14 on dist-test-slave-5k54"
server_key: "e2b5fb0821433fbb5e96e1ab7a879bce"
server_key_iv: "573e2f63733b3158ee63e74e4edfa1c1"
server_key_version: "encryptionkey@0"
I20260808 18:27:14.326648 15992 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 18:27:14.329049 16009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:14.330272 15992 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:27:14.330371 15992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-2/data,/tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-2/wal
uuid: "ed9e33a81d66483dafc2dcb7572b3e6e"
format_stamp: "Formatted at 2026-08-08 18:27:14 on dist-test-slave-5k54"
server_key: "e2b5fb0821433fbb5e96e1ab7a879bce"
server_key_iv: "573e2f63733b3158ee63e74e4edfa1c1"
server_key_version: "encryptionkey@0"
I20260808 18:27:14.330466 15992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:14.366243 15992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:14.369123 15992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:14.369282 15992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:14.376003 15992 rpc_server.cc:307] RPC server started. Bound to: 127.23.72.188:46559
I20260808 18:27:14.376291 16063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.72.188:46559 every 8 connection(s)
I20260808 18:27:14.376969 15992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-2/data/info.pb
I20260808 18:27:14.379686 16064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:14.383267 16064 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40887 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:27:14.383759 15903 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.72.188" port: 46559 } attempt: 3
I20260808 18:27:14.385703 23842 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVyYox2/build/debug/bin/kudu as pid 15992
I20260808 18:27:14.385834 23842 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVyYox2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213345431840-23842-0/minicluster-data/master-2/wal/instance
I20260808 18:27:14.386185 23842 external_mini_cluster.cc:1428] Setting key c89fd1220b69159174bccb8150adb1e4
I20260808 18:27:14.391772 15984 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.72.188" port: 46559 } attempt: 3
Aug 08 18:27:14 dist-test-slave-5k54 krb5kdc[15815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213633, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.190@KRBTEST.COM for kudu/127.23.72.188@KRBTEST.COM
Aug 08 18:27:14 dist-test-slave-5k54 krb5kdc[15815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213634, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.188@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
Aug 08 18:27:14 dist-test-slave-5k54 krb5kdc[15815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.72.188@KRBTEST.COM
Aug 08 18:27:14 dist-test-slave-5k54 krb5kdc[15815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213634, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.189@KRBTEST.COM for kudu/127.23.72.188@KRBTEST.COM
I20260808 18:27:14.397935 15906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:14.385202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50028 (local address 127.23.72.190:40887)
0808 18:27:14.385332 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:14.385337 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:14.385690 (+ 353us) server_negotiation.cc:407] Connection header received
0808 18:27:14.386529 (+ 839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:14.386535 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:14.386597 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:14.386688 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:14.388176 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:14.388890 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:14.390110 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:14.390434 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:14.390523 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:14.393829 (+ 3306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:14.393832 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:14.393836 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:14.393881 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:14.395719 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:14.396346 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:14.396349 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:14.396351 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:14.396435 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:14.396801 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:14.396806 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:14.396807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:14.397108 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:14.397185 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:14.397642 (+ 457us) server_negotiation.cc:299] Negotiation successful
0808 18:27:14.397763 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 18:27:14.399072 16066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:14.384924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53502 (local address 127.23.72.188:46559)
0808 18:27:14.385296 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:14.385321 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:14.385382 (+ 61us) server_negotiation.cc:407] Connection header received
0808 18:27:14.386294 (+ 912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:14.386318 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:14.386563 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:14.386749 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:14.387215 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:14.388515 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:14.389420 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:14.390110 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:14.390249 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:14.392996 (+ 2747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:14.393004 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:14.393036 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:14.393064 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:14.395893 (+ 2829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:14.396383 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:14.396391 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:14.396396 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:14.396527 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:14.396818 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:14.396821 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:14.396822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:14.397257 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:14.397378 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:14.397809 (+ 431us) server_negotiation.cc:299] Negotiation successful
0808 18:27:14.398096 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":246,"thread_start_us":125,"threads_started":1}
I20260808 18:27:14.399657 15907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:14.384246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:46559 (local address 127.0.0.1:53502)
0808 18:27:14.384379 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:14.384396 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:14.384491 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:14.386736 (+ 2245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:14.386739 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:14.386960 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:14.387068 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:14.387076 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:14.388616 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:14.388619 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:14.389294 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:14.389305 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:14.390207 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:14.390211 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:14.390300 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:14.390961 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:14.390973 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:14.392822 (+ 1849us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:14.396012 (+ 3190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:14.396015 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:14.396018 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:14.396255 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:14.396629 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:14.396632 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:14.396634 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:14.396716 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:14.397426 (+ 710us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:14.397429 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:14.397517 (+ 88us) client_negotiation.cc:769] Sending connection context
0808 18:27:14.397618 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 18:27:14.399545 (+ 1927us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":1772}
I20260808 18:27:14.400295 16070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:14.386919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53518 (local address 127.23.72.188:46559)
0808 18:27:14.387208 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:14.387223 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:14.387304 (+ 81us) server_negotiation.cc:407] Connection header received
0808 18:27:14.387492 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:14.387494 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:14.387539 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:14.387639 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:14.388543 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:14.389133 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:14.390045 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:14.390392 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:14.390508 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:14.395187 (+ 4679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:14.395191 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:14.395194 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:14.395218 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:14.397315 (+ 2097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:14.397850 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:14.397853 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:14.397855 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:14.397940 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:14.398269 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:14.398273 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:14.398274 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:14.398765 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:14.399844 (+ 1079us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:14.399924 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 18:27:14.400101 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":89,"threads_started":1}
I20260808 18:27:14.399082 16067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:14.385170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.190:40887 (local address 127.0.0.1:50028)
0808 18:27:14.385555 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:14.385600 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:14.386303 (+ 703us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:14.387200 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:14.387206 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:14.387841 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:14.388021 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:14.388033 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:14.389019 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:14.389023 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:14.389962 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:14.389972 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:14.390672 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:14.390675 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:14.390816 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:14.391356 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:14.391375 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:14.393663 (+ 2288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:14.395851 (+ 2188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:14.395869 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:14.395879 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:14.396228 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:14.396549 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:14.396555 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:14.396560 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:14.396692 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:14.397266 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:14.397271 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:14.397411 (+ 140us) client_negotiation.cc:769] Sending connection context
0808 18:27:14.398330 (+ 919us) client_negotiation.cc:240] Negotiation successful
0808 18:27:14.398496 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":156,"threads_started":1}
I20260808 18:27:14.402081 15986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:14.392255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:46559 (local address 127.0.0.1:53528)
0808 18:27:14.392421 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:14.392438 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:14.392531 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:14.393008 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:14.393011 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:14.393211 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:14.393317 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:14.393324 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:14.394431 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:14.394437 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:14.395267 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:14.395277 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:14.395991 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:14.395997 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:14.396093 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:14.396537 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:14.396553 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:14.397984 (+ 1431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:14.400300 (+ 2316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:14.400306 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:14.400311 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:14.400564 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:14.400895 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:14.400898 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:14.400900 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:14.401002 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:14.401602 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:14.401609 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:14.401708 (+ 99us) client_negotiation.cc:769] Sending connection context
0808 18:27:14.401793 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 18:27:14.401929 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260808 18:27:14.402070 16064 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46103 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:27:14.403028 16071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:14.392423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53528 (local address 127.23.72.188:46559)
0808 18:27:14.392736 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:14.392745 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:14.392759 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:27:14.392804 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:14.392807 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:14.392845 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:14.392989 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:14.393456 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:14.394291 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:14.395408 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:14.395828 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:14.396294 (+ 466us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:14.398120 (+ 1826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:14.398124 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:14.398126 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:14.398148 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:14.400180 (+ 2032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:14.400691 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:14.400694 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:14.400695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:14.400790 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:14.401114 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:14.401117 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:14.401118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:14.401485 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:14.402532 (+ 1047us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:14.402605 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 18:27:14.402853 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":97,"threads_started":1}
I20260808 18:27:14.404497 15903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b2e9b5598014afd898b7854e73f5172: Bootstrap starting.
I20260808 18:27:14.406155 15984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4b0579168b34515b33db59f918997da: Bootstrap starting.
Aug 08 18:27:14 dist-test-slave-5k54 krb5kdc[15815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213634, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.188@KRBTEST.COM for kudu/127.23.72.189@KRBTEST.COM
I20260808 18:27:14.408349 15903 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b2e9b5598014afd898b7854e73f5172: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:14.408861 15984 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4b0579168b34515b33db59f918997da: Neither blocks nor log segments found. Creating new log.
Aug 08 18:27:14 dist-test-slave-5k54 krb5kdc[15815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.72.190@KRBTEST.COM
I20260808 18:27:14.409794 15984 log.cc:824] T 00000000000000000000000000000000 P c4b0579168b34515b33db59f918997da: Log is configured to *not* fsync() on all Append() calls
I20260808 18:27:14.409833 15903 log.cc:824] T 00000000000000000000000000000000 P 4b2e9b5598014afd898b7854e73f5172: Log is configured to *not* fsync() on all Append() calls
I20260808 18:27:14.411991 16067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:14.402508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.189:46103 (local address 127.0.0.1:48784)
0808 18:27:14.402712 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:14.402722 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:14.402810 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:14.403053 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:14.403055 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:14.403240 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:14.403346 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:14.403352 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:14.404357 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:14.404362 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:14.405248 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:14.405260 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:14.405985 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:14.405988 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:14.406083 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:14.406553 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:14.406567 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:14.408093 (+ 1526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:14.410198 (+ 2105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:14.410202 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:14.410204 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:14.410474 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:14.410816 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:14.410818 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:14.410820 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:14.410934 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:14.411471 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:14.411474 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:14.411569 (+ 95us) client_negotiation.cc:769] Sending connection context
0808 18:27:14.411675 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 18:27:14.411808 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150}
I20260808 18:27:14.412113 15903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b2e9b5598014afd898b7854e73f5172: No bootstrap required, opened a new log
I20260808 18:27:14.412533 15990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:14.402710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48784 (local address 127.23.72.189:46103)
0808 18:27:14.402847 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:14.402853 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:14.402878 (+ 25us) server_negotiation.cc:407] Connection header received
0808 18:27:14.402911 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:14.402913 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:14.402955 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:14.403040 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:14.403485 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:14.404221 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:14.405376 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:14.405722 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:14.405798 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:14.408248 (+ 2450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:14.408252 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:14.408255 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:14.408278 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:14.409998 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:14.410601 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:14.410605 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:14.410606 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:14.410691 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:14.411022 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:14.411027 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:14.411028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:14.411311 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:14.411454 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:14.412260 (+ 806us) server_negotiation.cc:299] Negotiation successful
0808 18:27:14.412376 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 18:27:14.413260 16064 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.72.188" port: 46559 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:27:14.413920 15984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4b0579168b34515b33db59f918997da: No bootstrap required, opened a new log
I20260808 18:27:14.414757 15906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:14.402360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50034 (local address 127.23.72.190:40887)
0808 18:27:14.402456 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:14.402459 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:14.403881 (+ 1422us) server_negotiation.cc:407] Connection header received
0808 18:27:14.403942 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:14.403944 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:14.403984 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:14.404050 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:14.404857 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:14.405551 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:14.406673 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:14.406983 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:14.407074 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:14.409981 (+ 2907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:14.409985 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:14.409987 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:14.410011 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:14.412149 (+ 2138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:14.412908 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:14.412912 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:14.412913 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:14.413009 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:14.413330 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:14.413333 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:14.413335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:14.413555 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:14.414424 (+ 869us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:14.414504 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 18:27:14.414620 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 18:27:14.416661 15903 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b2e9b5598014afd898b7854e73f5172 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b2e9b5598014afd898b7854e73f5172" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40887 } } peers { permanent_uuid: "c4b0579168b34515b33db59f918997da" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46103 } } peers { permanent_uuid: "ed9e33a81d66483dafc2dcb7572b3e6e" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 46559 } }
I20260808 18:27:14.416887 15903 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b2e9b5598014afd898b7854e73f5172 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b2e9b5598014afd898b7854e73f5172, State: Initialized, Role: FOLLOWER
I20260808 18:27:14.417212 15903 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b2e9b5598014afd898b7854e73f5172 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b2e9b5598014afd898b7854e73f5172" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40887 } } peers { permanent_uuid: "c4b0579168b34515b33db59f918997da" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46103 } } peers { permanent_uuid: "ed9e33a81d66483dafc2dcb7572b3e6e" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 46559 } }
I20260808 18:27:14.418264 15984 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4b0579168b34515b33db59f918997da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b2e9b5598014afd898b7854e73f5172" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40887 } } peers { permanent_uuid: "c4b0579168b34515b33db59f918997da" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46103 } } peers { permanent_uuid: "ed9e33a81d66483dafc2dcb7572b3e6e" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 46559 } }
I20260808 18:27:14.418488 15984 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4b0579168b34515b33db59f918997da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b0579168b34515b33db59f918997da, State: Initialized, Role: FOLLOWER
I20260808 18:27:14.418711 15903 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b2e9b5598014afd898b7854e73f5172 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:27:14.418790 15984 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4b0579168b34515b33db59f918997da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b2e9b5598014afd898b7854e73f5172" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40887 } } peers { permanent_uuid: "c4b0579168b34515b33db59f918997da" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46103 } } peers { permanent_uuid: "ed9e33a81d66483dafc2dcb7572b3e6e" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 46559 } }
Aug 08 18:27:14 dist-test-slave-5k54 krb5kdc[15815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213634, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.72.188@KRBTEST.COM for kudu/127.23.72.188@KRBTEST.COM
I20260808 18:27:14.420194 15984 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4b0579168b34515b33db59f918997da [sys.catalog]: configured and running, proceeding with master startup.
Aug 08 18:27:14 dist-test-slave-5k54 krb5kdc[15815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.72.189@KRBTEST.COM
I20260808 18:27:14.424346 16071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:14.413911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53530 (local address 127.23.72.188:46559)
0808 18:27:14.414016 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:14.414022 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:14.414176 (+ 154us) server_negotiation.cc:407] Connection header received
0808 18:27:14.414310 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:14.414313 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:14.414348 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:14.414391 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:14.415055 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:14.415834 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:14.416822 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:14.417149 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:14.417224 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:14.420234 (+ 3010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:14.420238 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:14.420241 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:14.420265 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:14.422042 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:14.422543 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:14.422546 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:14.422548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:14.422628 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:14.423186 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:14.423188 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:14.423191 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:14.423664 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:14.423739 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:14.424083 (+ 344us) server_negotiation.cc:299] Negotiation successful
0808 18:27:14.424195 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 18:27:14.424940 16067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:14.413919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.72.188:46559 (local address 127.0.0.1:53530)
0808 18:27:14.414115 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:14.414130 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:14.414224 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:14.414500 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:14.414503 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:14.414666 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:14.414739 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:14.414745 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:14.415970 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:14.415974 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:14.416702 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:14.416713 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:14.417523 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:14.417529 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:14.417606 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:14.418072 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:14.418085 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:14.419820 (+ 1735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:14.422212 (+ 2392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:14.422214 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:14.422216 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:14.422441 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:14.422725 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:14.422980 (+ 255us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:14.422983 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:14.423080 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:14.423806 (+ 726us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:14.423812 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:14.423903 (+ 91us) client_negotiation.cc:769] Sending connection context
0808 18:27:14.424654 (+ 751us) client_negotiation.cc:240] Negotiation successful
0808 18:27:14.424801 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139}
I20260808 18:27:14.426430 15990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:14.415981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48796 (local address 127.23.72.189:46103)
0808 18:27:14.416321 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:14.416325 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:14.416337 (+ 12us) server_negotiation.cc:407] Connection header received
0808 18:27:14.416383 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:14.416385 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:14.416422 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:14.416490 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:14.417277 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:14.417992 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:14.419046 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:14.419354 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:14.419477 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:14.422481 (+ 3004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:14.422484 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:14.422487 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:14.422512 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:14.424177 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:14.424657 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:14.424661 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:14.424663 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:14.424749 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:14.425079 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:14.425082 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:14.425084 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:14.425278 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:14.425476 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:14.426173 (+ 697us) server_negotiation.cc:299] Negotiation successful
0808 18:27:14.426278 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274}
I20260808 18:27:14.428943 16064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed9e33a81d66483dafc2dcb7572b3e6e: Bootstrap starting.
W20260808 18:27:14.429488 16099 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c4b0579168b34515b33db59f918997da: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:27:14.429576 16099 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:27:14.429788 16075 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4b0579168b34515b33db59f918997da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b2e9b5598014afd898b7854e73f5172" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40887 } } peers { permanent_uuid: "c4b0579168b34515b33db59f918997da" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46103 } } peers { permanent_uuid: "ed9e33a81d66483dafc2dcb7572b3e6e" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 46559 } } }
I20260808 18:27:14.429917 16075 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4b0579168b34515b33db59f918997da [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:27:14.428248 15911 master_runner.cc:186] Error getting master registration for 127.23.72.188:46559: OK, instance_id { permanent_uuid: "ed9e33a81d66483dafc2dcb7572b3e6e" instance_seqno: 1786213634374115 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 18:27:14.431401 15911 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 18:27:14.431659 16064 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed9e33a81d66483dafc2dcb7572b3e6e: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:14.432292 16064 log.cc:824] T 00000000000000000000000000000000 P ed9e33a81d66483dafc2dcb7572b3e6e: Log is configured to *not* fsync() on all Append() calls
I20260808 18:27:14.434209 16064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed9e33a81d66483dafc2dcb7572b3e6e: No bootstrap required, opened a new log
I20260808 18:27:14.427049 15828 master_runner.cc:186] Error getting master registration for 127.23.72.188:46559: OK, instance_id { permanent_uuid: "ed9e33a81d66483dafc2dcb7572b3e6e" instance_seqno: 1786213634374115 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 18:27:14.435809 15828 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 18:27:14.436493 23842 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:27:14.437839 15875 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "4b2e9b5598014afd898b7854e73f5172"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50034
I20260808 18:27:14.438073 15875 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b2e9b5598014afd898b7854e73f5172 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 18:27:14.438195 15875 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b2e9b5598014afd898b7854e73f5172 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:14.438508 16064 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed9e33a81d66483dafc2dcb7572b3e6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b2e9b5598014afd898b7854e73f5172" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40887 } } peers { permanent_uuid: "c4b0579168b34515b33db59f918997da" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46103 } } peers { permanent_uuid: "ed9e33a81d66483dafc2dcb7572b3e6e" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 46559 } }
I20260808 18:27:14.438733 16064 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed9e33a81d66483dafc2dcb7572b3e6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed9e33a81d66483dafc2dcb7572b3e6e, State: Initialized, Role: FOLLOWER
I20260808 18:27:14.439054 16064 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed9e33a81d66483dafc2dcb7572b3e6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b2e9b5598014afd898b7854e73f5172" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40887 } } peers { permanent_uuid: "c4b0579168b34515b33db59f918997da" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46103 } } peers { permanent_uuid: "ed9e33a81d66483dafc2dcb7572b3e6e" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 46559 } }
I20260808 18:27:14.439584 15875 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b2e9b5598014afd898b7854e73f5172 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b2e9b5598014afd898b7854e73f5172" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40887 } } peers { permanent_uuid: "c4b0579168b34515b33db59f918997da" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46103 } } peers { permanent_uuid: "ed9e33a81d66483dafc2dcb7572b3e6e" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 46559 } }
I20260808 18:27:14.439630 16103 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed9e33a81d66483dafc2dcb7572b3e6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b2e9b5598014afd898b7854e73f5172" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40887 } } peers { permanent_uuid: "c4b0579168b34515b33db59f918997da" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46103 } } peers { permanent_uuid: "ed9e33a81d66483dafc2dcb7572b3e6e" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 46559 } } }
I20260808 18:27:14.439880 16103 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed9e33a81d66483dafc2dcb7572b3e6e [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:27:14.439946 16074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b2e9b5598014afd898b7854e73f5172 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b2e9b5598014afd898b7854e73f5172" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40887 } } peers { permanent_uuid: "c4b0579168b34515b33db59f918997da" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46103 } } peers { permanent_uuid: "ed9e33a81d66483dafc2dcb7572b3e6e" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 46559 } } }
I20260808 18:27:14.440065 16074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b2e9b5598014afd898b7854e73f5172 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:27:14.440402 16064 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed9e33a81d66483dafc2dcb7572b3e6e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:27:14.440533 15875 leader_election.cc:288] T 00000000000000000000000000000000 P 4b2e9b5598014afd898b7854e73f5172 [CANDIDATE]: Term 1 election: Requested vote from peers c4b0579168b34515b33db59f918997da (127.23.72.189:46103), ed9e33a81d66483dafc2dcb7572b3e6e (127.23.72.188:46559)
I20260808 18:27:14.440976 15958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4b2e9b5598014afd898b7854e73f5172" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c4b0579168b34515b33db59f918997da"
I20260808 18:27:14.441212 15958 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4b0579168b34515b33db59f918997da [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:14.442637 15958 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c4b0579168b34515b33db59f918997da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b2e9b5598014afd898b7854e73f5172 in term 1.
I20260808 18:27:14.443204 16037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4b2e9b5598014afd898b7854e73f5172" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ed9e33a81d66483dafc2dcb7572b3e6e"
I20260808 18:27:14.443646 15845 leader_election.cc:302] T 00000000000000000000000000000000 P 4b2e9b5598014afd898b7854e73f5172 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b2e9b5598014afd898b7854e73f5172, c4b0579168b34515b33db59f918997da; no voters:
I20260808 18:27:14.443915 16074 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b2e9b5598014afd898b7854e73f5172 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:27:14.443418 16037 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed9e33a81d66483dafc2dcb7572b3e6e [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:14.444303 16074 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b2e9b5598014afd898b7854e73f5172 [term 1 LEADER]: Becoming Leader. State: Replica: 4b2e9b5598014afd898b7854e73f5172, State: Running, Role: LEADER
I20260808 18:27:14.444860 16074 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b2e9b5598014afd898b7854e73f5172 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b2e9b5598014afd898b7854e73f5172" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40887 } } peers { permanent_uuid: "c4b0579168b34515b33db59f918997da" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46103 } } peers { permanent_uuid: "ed9e33a81d66483dafc2dcb7572b3e6e" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 46559 } }
I20260808 18:27:14.445262 16037 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ed9e33a81d66483dafc2dcb7572b3e6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b2e9b5598014afd898b7854e73f5172 in term 1.
W20260808 18:27:14.445510 16114 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ed9e33a81d66483dafc2dcb7572b3e6e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:27:14.445591 16114 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:27:14.447256 16074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b2e9b5598014afd898b7854e73f5172 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b2e9b5598014afd898b7854e73f5172. Latest consensus state: current_term: 1 leader_uuid: "4b2e9b5598014afd898b7854e73f5172" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b2e9b5598014afd898b7854e73f5172" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40887 } } peers { permanent_uuid: "c4b0579168b34515b33db59f918997da" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46103 } } peers { permanent_uuid: "ed9e33a81d66483dafc2dcb7572b3e6e" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 46559 } } }
I20260808 18:27:14.447516 16074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b2e9b5598014afd898b7854e73f5172 [sys.catalog]: This master's current role is: LEADER
W20260808 18:27:14.451500 16086 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4b2e9b5598014afd898b7854e73f5172: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:27:14.451570 16086 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:27:14.451730 15992 master_runner.cc:263] No leader master found from master ed9e33a81d66483dafc2dcb7572b3e6e
I20260808 18:27:14.451792 15992 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 18:27:14.461161 16122 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:27:14.462275 16122 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:27:14.468585 15990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:14.455846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48812 (local address 127.23.72.189:46103)
0808 18:27:14.455963 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:14.455968 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:14.456377 (+ 409us) server_negotiation.cc:407] Connection header received
0808 18:27:14.456585 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:14.456591 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:14.456646 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:14.456695 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:14.457671 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:14.458442 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:14.459662 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:14.459973 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:14.460049 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:14.462231 (+ 2182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:14.462235 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:14.462237 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:14.462263 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:14.464045 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:14.464576 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:14.464580 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:14.464582 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:14.464670 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:14.467151 (+ 2481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:14.467155 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:14.467157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:14.467380 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:14.467507 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:14.468293 (+ 786us) server_negotiation.cc:299] Negotiation successful
0808 18:27:14.468415 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 18:27:14.470758 16071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:14.456153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53544 (local address 127.23.72.188:46559)
0808 18:27:14.456263 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:14.456268 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:14.456649 (+ 381us) server_negotiation.cc:407] Connection header received
0808 18:27:14.456811 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:14.456814 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:14.456856 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:14.456906 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:14.457798 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:14.458557 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:14.460091 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:14.460423 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:14.460495 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:14.465272 (+ 4777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:14.465276 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:14.465279 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:14.465304 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:14.467019 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:14.469068 (+ 2049us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:14.469072 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:14.469073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:14.469175 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:14.469508 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:14.469511 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:14.469513 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:14.470010 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:14.470142 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:14.470485 (+ 343us) server_negotiation.cc:299] Negotiation successful
0808 18:27:14.470608 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 18:27:14.476363 15958 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c4b0579168b34515b33db59f918997da [term 1 FOLLOWER]: Refusing update from remote peer 4b2e9b5598014afd898b7854e73f5172: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:27:14.477166 16037 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ed9e33a81d66483dafc2dcb7572b3e6e [term 1 FOLLOWER]: Refusing update from remote peer 4b2e9b5598014afd898b7854e73f5172: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:27:14.477941 16116 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4b2e9b5598014afd898b7854e73f5172 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed9e33a81d66483dafc2dcb7572b3e6e" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 46559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:27:14.478399 16116 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4b2e9b5598014afd898b7854e73f5172 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4b0579168b34515b33db59f918997da" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:27:14.480366 15906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:14.456039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50040 (local address 127.23.72.190:40887)
0808 18:27:14.456138 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:14.456142 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:14.460017 (+ 3875us) server_negotiation.cc:407] Connection header received
0808 18:27:14.462641 (+ 2624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:14.462645 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:14.462714 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:14.462782 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:14.464424 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:14.465222 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:14.468843 (+ 3621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:14.469193 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:14.469264 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:14.476511 (+ 7247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:14.476515 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:14.476518 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:14.476547 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:14.478396 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:14.478935 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:14.478939 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:14.478941 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:14.479025 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:14.479316 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:14.479321 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:14.479322 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:14.479622 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:14.479747 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:14.480109 (+ 362us) server_negotiation.cc:299] Negotiation successful
0808 18:27:14.480224 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 18:27:14.482357 16103 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed9e33a81d66483dafc2dcb7572b3e6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b2e9b5598014afd898b7854e73f5172. Latest consensus state: current_term: 1 leader_uuid: "4b2e9b5598014afd898b7854e73f5172" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b2e9b5598014afd898b7854e73f5172" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40887 } } peers { permanent_uuid: "c4b0579168b34515b33db59f918997da" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46103 } } peers { permanent_uuid: "ed9e33a81d66483dafc2dcb7572b3e6e" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 46559 } } }
I20260808 18:27:14.482491 16103 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed9e33a81d66483dafc2dcb7572b3e6e [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:27:14.486676 16075 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4b0579168b34515b33db59f918997da [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b2e9b5598014afd898b7854e73f5172. Latest consensus state: current_term: 1 leader_uuid: "4b2e9b5598014afd898b7854e73f5172" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b2e9b5598014afd898b7854e73f5172" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40887 } } peers { permanent_uuid: "c4b0579168b34515b33db59f918997da" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46103 } } peers { permanent_uuid: "ed9e33a81d66483dafc2dcb7572b3e6e" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 46559 } } }
I20260808 18:27:14.486814 16075 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4b0579168b34515b33db59f918997da [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:27:14.491238 16116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b2e9b5598014afd898b7854e73f5172 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4b2e9b5598014afd898b7854e73f5172" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b2e9b5598014afd898b7854e73f5172" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40887 } } peers { permanent_uuid: "c4b0579168b34515b33db59f918997da" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46103 } } peers { permanent_uuid: "ed9e33a81d66483dafc2dcb7572b3e6e" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 46559 } } }
I20260808 18:27:14.491391 16116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b2e9b5598014afd898b7854e73f5172 [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:14.491526 16127 mvcc.cc:205] Tried to move back new op lower bound from 7316331046805860352 to 7316331046692253696. Current Snapshot: MvccSnapshot[applied={T|T < 7316331046805860352}]
I20260808 18:27:14.492733 16103 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed9e33a81d66483dafc2dcb7572b3e6e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4b2e9b5598014afd898b7854e73f5172" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b2e9b5598014afd898b7854e73f5172" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40887 } } peers { permanent_uuid: "c4b0579168b34515b33db59f918997da" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46103 } } peers { permanent_uuid: "ed9e33a81d66483dafc2dcb7572b3e6e" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 46559 } } }
I20260808 18:27:14.492874 16103 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed9e33a81d66483dafc2dcb7572b3e6e [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:27:14.496415 16128 mvcc.cc:205] Tried to move back new op lower bound from 7316331046805860352 to 7316331046692253696. Current Snapshot: MvccSnapshot[applied={T|T < 7316331046805860352}]
I20260808 18:27:14.497121 16074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b2e9b5598014afd898b7854e73f5172 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4b2e9b5598014afd898b7854e73f5172" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b2e9b5598014afd898b7854e73f5172" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40887 } } peers { permanent_uuid: "c4b0579168b34515b33db59f918997da" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46103 } } peers { permanent_uuid: "ed9e33a81d66483dafc2dcb7572b3e6e" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 46559 } } }
I20260808 18:27:14.497233 16074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b2e9b5598014afd898b7854e73f5172 [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:14.498250 16075 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4b0579168b34515b33db59f918997da [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4b2e9b5598014afd898b7854e73f5172" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b2e9b5598014afd898b7854e73f5172" member_type: VOTER last_known_addr { host: "127.23.72.190" port: 40887 } } peers { permanent_uuid: "c4b0579168b34515b33db59f918997da" member_type: VOTER last_known_addr { host: "127.23.72.189" port: 46103 } } peers { permanent_uuid: "ed9e33a81d66483dafc2dcb7572b3e6e" member_type: VOTER last_known_addr { host: "127.23.72.188" port: 46559 } } }
I20260808 18:27:14.498368 16075 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4b0579168b34515b33db59f918997da [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:27:14.499643 16130 mvcc.cc:205] Tried to move back new op lower bound from 7316331046805860352 to 7316331046692253696. Current Snapshot: MvccSnapshot[applied={T|T < 7316331046805860352}]
I20260808 18:27:14.502314 16122 catalog_manager.cc:1347] Generated new cluster ID: 81de2982ab29444ea0f839bc1d49c6c3
I20260808 18:27:14.502388 16122 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:27:14.534318 16122 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:27:14.535719 16122 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:27:14.544864 16122 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b2e9b5598014afd898b7854e73f5172: Generated new TSK 0
I20260808 18:27:14.545710 16122 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 18:27:14.577029 15990 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:27:14.573901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48818 (local address 127.23.72.189:46103)
0808 18:27:14.574026 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:14.574030 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:14.575921 (+ 1891us) server_negotiation.cc:407] Connection header received
0808 18:27:14.576129 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:14.576134 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:14.576188 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:14.576243 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:14.576896 (+ 653us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48818: BlockingRecv error: recv got EOF from 127.0.0.1:48818 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260808 18:27:14.578743 16071 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:27:14.574678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53554 (local address 127.23.72.188:46559)
0808 18:27:14.574816 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:14.574821 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:14.577557 (+ 2736us) server_negotiation.cc:407] Connection header received
0808 18:27:14.577679 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:14.577683 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:14.577733 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:14.577822 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:14.578623 (+ 801us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53554: BlockingRecv error: recv got EOF from 127.0.0.1:53554 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260808 18:27:14.578769 15906 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:27:14.575343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50050 (local address 127.23.72.190:40887)
0808 18:27:14.575498 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:14.575502 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:14.577904 (+ 2402us) server_negotiation.cc:407] Connection header received
0808 18:27:14.577965 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:14.577968 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:14.578012 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:14.578090 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:14.578635 (+ 545us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50050: BlockingRecv error: recv got EOF from 127.0.0.1:50050 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260808 18:27:14.592473 15990 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:27:14.590433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48826 (local address 127.23.72.189:46103)
0808 18:27:14.590547 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:14.590552 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:14.591132 (+ 580us) server_negotiation.cc:407] Connection header received
0808 18:27:14.591397 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:14.591401 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:14.591606 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:14.591679 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:14.592358 (+ 679us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48826: BlockingRecv error: recv got EOF from 127.0.0.1:48826 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260808 18:27:14.597734 15990 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:27:14.596161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48828 (local address 127.23.72.189:46103)
0808 18:27:14.596278 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:14.596282 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:14.596804 (+ 522us) server_negotiation.cc:407] Connection header received
0808 18:27:14.596991 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:14.596994 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:14.597043 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:14.597094 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:14.597635 (+ 541us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48828: BlockingRecv error: recv got EOF from 127.0.0.1:48828 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260808 18:27:14.603499 15990 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:27:14.601902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48834 (local address 127.23.72.189:46103)
0808 18:27:14.602007 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:14.602011 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:14.602533 (+ 522us) server_negotiation.cc:407] Connection header received
0808 18:27:14.602729 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:14.602732 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:14.602785 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:14.602842 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:14.603374 (+ 532us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48834: BlockingRecv error: recv got EOF from 127.0.0.1:48834 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260808 18:27:14.611235 15990 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:27:14.609722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48836 (local address 127.23.72.189:46103)
0808 18:27:14.609853 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:14.609857 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:14.610361 (+ 504us) server_negotiation.cc:407] Connection header received
0808 18:27:14.610536 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:14.610539 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:14.610586 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:14.610634 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:14.611136 (+ 502us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48836: BlockingRecv error: recv got EOF from 127.0.0.1:48836 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260808 18:27:14.623523 15990 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:27:14.621743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48850 (local address 127.23.72.189:46103)
0808 18:27:14.621874 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:14.621878 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:14.622426 (+ 548us) server_negotiation.cc:407] Connection header received
0808 18:27:14.622668 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:14.622672 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:14.622725 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:14.622777 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:14.623366 (+ 589us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48850: BlockingRecv error: recv got EOF from 127.0.0.1:48850 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260808 18:27:14.643383 15990 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:27:14.641991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48854 (local address 127.23.72.189:46103)
0808 18:27:14.642130 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:14.642134 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:14.642423 (+ 289us) server_negotiation.cc:407] Connection header received
0808 18:27:14.642635 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:14.642641 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:14.642695 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:14.642789 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:14.643262 (+ 473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48854: BlockingRecv error: recv got EOF from 127.0.0.1:48854 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260808 18:27:14.678853 15990 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:27:14.677663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48868 (local address 127.23.72.189:46103)
0808 18:27:14.677833 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:14.677839 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:14.677975 (+ 136us) server_negotiation.cc:407] Connection header received
0808 18:27:14.678072 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:14.678075 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:14.678138 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:14.678243 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:14.678695 (+ 452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48868: BlockingRecv error: recv got EOF from 127.0.0.1:48868 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260808 18:27:14.746380 15990 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:27:14.745166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48884 (local address 127.23.72.189:46103)
0808 18:27:14.745314 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:14.745318 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:14.745469 (+ 151us) server_negotiation.cc:407] Connection header received
0808 18:27:14.745597 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:14.745600 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:14.745651 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:14.745734 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:14.746253 (+ 519us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48884: BlockingRecv error: recv got EOF from 127.0.0.1:48884 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260808 18:27:14.878134 15990 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:27:14.876897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48892 (local address 127.23.72.189:46103)
0808 18:27:14.877114 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:14.877119 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:14.877192 (+ 73us) server_negotiation.cc:407] Connection header received
0808 18:27:14.877353 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:14.877356 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:14.877409 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:14.877482 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:14.877957 (+ 475us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48892: BlockingRecv error: recv got EOF from 127.0.0.1:48892 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
W20260808 18:27:15.137437 15990 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:27:15.136352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48900 (local address 127.23.72.189:46103)
0808 18:27:15.136502 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:15.136506 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:15.136604 (+ 98us) server_negotiation.cc:407] Connection header received
0808 18:27:15.136734 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:15.136736 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:15.136789 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:15.136889 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:15.137328 (+ 439us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48900: BlockingRecv error: recv got EOF from 127.0.0.1:48900 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 18:27:15.430929 16099 catalog_manager.cc:1259] Loaded cluster ID: 81de2982ab29444ea0f839bc1d49c6c3
I20260808 18:27:15.431043 16099 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c4b0579168b34515b33db59f918997da: loading cluster ID for follower catalog manager: success
I20260808 18:27:15.432761 16099 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c4b0579168b34515b33db59f918997da: acquiring CA information for follower catalog manager: success
I20260808 18:27:15.434017 16099 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c4b0579168b34515b33db59f918997da: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 18:27:15.446825 16114 catalog_manager.cc:1259] Loaded cluster ID: 81de2982ab29444ea0f839bc1d49c6c3
I20260808 18:27:15.446898 16114 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ed9e33a81d66483dafc2dcb7572b3e6e: loading cluster ID for follower catalog manager: success
I20260808 18:27:15.448541 16114 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ed9e33a81d66483dafc2dcb7572b3e6e: acquiring CA information for follower catalog manager: success
I20260808 18:27:15.449653 16114 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ed9e33a81d66483dafc2dcb7572b3e6e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 18:27:15.655251 16203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:15.651553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48906 (local address 127.23.72.189:46103)
0808 18:27:15.651804 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:15.651808 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:15.651824 (+ 16us) server_negotiation.cc:407] Connection header received
0808 18:27:15.651967 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:15.651970 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:15.652100 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:15.652208 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:27:15.652467 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:15.653027 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:15.653900 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:15.654114 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:15.654188 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:15.654488 (+ 300us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:27:15.654911 (+ 423us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:27:15.655012 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:15.655083 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 18:27:15.655129 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":98,"threads_started":1}
I20260808 18:27:15.661789 16210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:15.658325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53558 (local address 127.23.72.188:46559)
0808 18:27:15.658519 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:15.658522 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:15.658539 (+ 17us) server_negotiation.cc:407] Connection header received
0808 18:27:15.658688 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:15.658690 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:15.658838 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:15.658939 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:27:15.659107 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:15.659671 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:15.660476 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:15.660708 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:15.660794 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:15.661048 (+ 254us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:27:15.661408 (+ 360us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:27:15.661494 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:15.661546 (+ 52us) server_negotiation.cc:299] Negotiation successful
0808 18:27:15.661590 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":63,"threads_started":1}
I20260808 18:27:15.669103 16203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:15.664652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48922 (local address 127.23.72.189:46103)
0808 18:27:15.664785 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:15.664789 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:15.665037 (+ 248us) server_negotiation.cc:407] Connection header received
0808 18:27:15.665247 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:15.665251 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:15.665314 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:15.665364 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:27:15.665868 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:15.666508 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:15.667900 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:15.668154 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:15.668263 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:15.668594 (+ 331us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:27:15.668675 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:27:15.668753 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:15.668909 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 18:27:15.668960 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 18:27:15.669149 16210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:15.665093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53566 (local address 127.23.72.188:46559)
0808 18:27:15.665217 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:15.665221 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:15.665576 (+ 355us) server_negotiation.cc:407] Connection header received
0808 18:27:15.665715 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:15.665720 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:15.665770 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:15.665854 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:27:15.666194 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:15.666944 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:15.667981 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:15.668304 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:15.668412 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:15.668670 (+ 258us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:27:15.668776 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:27:15.668851 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:15.668981 (+ 130us) server_negotiation.cc:299] Negotiation successful
0808 18:27:15.669034 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 18:27:15.669840 16217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:15.664637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50060 (local address 127.23.72.190:40887)
0808 18:27:15.664946 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:15.664950 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:15.664965 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:27:15.665108 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:15.665111 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:15.665285 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:15.665419 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:27:15.665688 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:15.666578 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:15.667911 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:15.668277 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:15.668631 (+ 354us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:15.669048 (+ 417us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:27:15.669376 (+ 328us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:27:15.669471 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:15.669631 (+ 160us) server_negotiation.cc:299] Negotiation successful
0808 18:27:15.669701 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":104,"threads_started":1}
I20260808 18:27:15.672772 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 15828
I20260808 18:27:15.682473 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 15911
I20260808 18:27:15.689343 23842 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVyYox2/build/debug/bin/kudu with pid 15992
2026-08-08T18:27:15Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3159 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6316 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (290251 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST